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(57) A high-speed vision sensor includes: an ana- 
log-to-digital converter array 13, in which one analog- 
to-digital converter 21 0 is provided in correspondence 
with all the photodetector elements 1 20 that are located 
on each row in a photodetector array 11; a parallel 
processing system 14 that includes processor elements 
400 and shift registers 410, both of which form a one- 



to-one correspondence with the photodetector ele- 
ments 120; and data buses 17, 18 and data buffers 19 
and 20 for data transfer to processing elements 400. 
The processing elements 400 perform high-speed im- 
age processing between adjacent pixels by parallel 
processings. By using the data buses 17, 18, it is pos- 
sible to attain, at a high rate of speed, such calculation 
processing that requires data supplied from outside. 
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Description 

TECHNICAL FIELD 

5 [0001] The present invention relates to a high-speed vision sensor having an image processing function. 
BACKGROUND ART 

[0002] High-speed image processing is required to operate high-speed robots in a factory automation (FA) system 
10 or the like. For example, when a robot is configured with a feedback loop between its vision sensor and its actuator, 
the actuator can be controlled in units of milliseconds. Accordingly, the image processing speed that matches this 
speed is required. However, because the image processing speed in modern vision systems is limited by the video 
frame rate, the robot can only operate at low speeds that match this image processing speed. It is impossible to take 
full advantage of the robot's performance. 
15 [0003] There are some high-speed CCD cameras that can take images at intervals of about one (1 ) millisecond. In 
these devices, the images are temporarily stored in memory and later read and processed. Therefore, the devices can 
be applied to such applications as image analysis. However, the devices have almost no real-time capability, and are 
not suitable for controlling robots or similar real-time applications. 

[0004] In order to overcome these problems, institutes such as the Massachusetts Institute of Technology, the Cal- 

20 jfornia Institute of Technology, and Mitsubishi Electric Corporation have been researching a vision chip that combines 
the image reading portion and the processing portion into one unit ("An Object Position and Orientation IC with Em- 
bedded Imager," David L. Standley, Solid State Circuits, Vol. 26, No. 1 2, Dec. 1 991 , pp. 1 853-1 859, IEEE); "Computing 
Motion Using Analog and Binary Resistive Networks," James Hutchinson, et al., Computer, Vol. 21, March 1988, pp. 
52-64, IEEE); and "Artificial Retinas — fast versatile image processors," Kazuo Kyuma, et al., Nature, Vol. 272, No- 

25 vember 10, 1994). However, these chips employ a fixed analog circuit that is easy to integrate. Accordingly, these 
circuits have various shortcomings. That is, the circuits require subsequent-processing of output signals. The type of 
image processing they can perform is limited to special applications. The circuits have a lack of universality. 
[0005] Japanese Unexamined Patent Application Publication No. HEI-1 0-145680 has proposed a vision chip that is 
capable of performing universal image processing. This vision chip is provided with a processing element for each 

30 photodetector. An analog-to-digital converter is provided for each photodetector row. The vision chip can reduce the 
processing time through parallel processing. The vision chip can also reduce the number of transmission lines between 
the photodetectors and the processing elements, achieving an optimal integration level for both. 
[0006] It is noted that the center of gravity (first order moment) of an image is information that is required in many 
image processings. In order to calculate the center of gravity of an image, it is necessary to perform calculation onto 

35 image data of each pixel and positional information (x direction positional information and y direction positional infor- 
mation) of the subject pixel. It is therefore necessary to store, in advance, positional information of each pixel in a 
memory. When using the technology in the Japanese Unexamined Patent Application Publication No. HEI-1 0-1 45680, 
it is necessary to transfer, in advance, positional information of respective pixels from a control circuit to the corre- 
sponding processing elements in succession. Each processing element performs calculation onto image data that is 

40 received from a corresponding photodetector and its positional data, and outputs the calculated result. It requires a lot 
of time to transfer the positional information to the processing elements. It is desirable to enhance the speed of such 
a basic image processing operation. 

DISCLOSURE OF THE INVENTION 

45 

[0007] In view of the foregoing, it is an object of the present invention to provide a multi-pixel high-speed vision 
sensor which has a simple circuit construction and which is capable of performing basic image processing at a high 
speed. 

[0008] In order to attain the above-described object, the present invention provides a high-speed vision sensor, 
so comprising: a photodetector array having a plurality of photodetectors, which are arranged two-dimensionally in a 
plurality of rows and in a plurality of columns; an analog-to-digital converter array having a plurality of analog-to-digital 
converters which are arranged one-dimensionally such that each analog-to-digital converter corresponds to one row 
in the photodetector array, each analog-to-digital converter converting, into digital signals, analog signals which are 
successively outputted from the photodetectors in the corresponding row; a parallel processing system including a 
55 parallel processing element array, the parallel processing element array having a plurality of processing elements which 
are arranged two-dimensionally in a plurality of rows and in a plurality of columns and in one-to-one correspondence 
with the plurality of photodetectors in the photodetector array, each processing element performing a predetermined 
calculation on digital signals which are transferred from the analog-to-digital converter array; a column-direction data- 
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transfer bus including a plurality of column-direction data-transfer data lines which are arranged in one to one corre- 
spondence with the plurality of columns in the parallel processing system, each column-direction data-transfer data 
line being connected to the processing elements that are located in the corresponding column and performing data 
transfer operation with each processing element in the corresponding column; a row-direction data-transfer bus in- 

5 eluding a plurality of row-direction data-transfer data lines which are arranged in one to one correspondence with the 
plurality of rows in the parallel processing system, each row-direction data-transfer data line being connected to the 
processing elements that are located in the corresponding row and performing data transfer operation with each 
processing element in the corresponding row; and a control circuit controlling the photodetector array, the analog-to- 
digital converter array, the parallel processing system, the column-direction data-transfer bus, and the row-direction 

10 data-transfer bus. 

[0009] According to the present invention, the plurality of processing elements are provided in one to one corre- 
spondence with the plurality of photodetectors. It is therefore possible to perform high-speed image processing oper- 
ation by parallel processing. Additionally, the analog-to-digital converter is provided to each row. Accordingly, the total 
number of transmission paths can be reduced. 

15 [0010] Additionally, there are provided, with respect to each processing element, a data bus which is dedicated to 
transferring data in a row direction, and another data bus which is dedicated to transferring data in a column direction. 
It is therefore possible to perform various image processing calculations. It is possible to attain a flexible processing 
performance while maintaining the architecture suitable for integration of the semiconductor elements. 
[0011] For example, it is preferable that the control circuit includes a center- of -gravity calculation control portion for 

20 controlling each column -direction data-transfer data line to perform data transfer operation to transfer positional infor- 
mation of the corresponding column to the processing elements on the corresponding column, for controlling each 
row-direction data-transfer data line to perform data transfer operation to transfer positional information of the corre- 
sponding row to the processing elements on the corresponding row, and for controlling each processing element to 
perform a predetermined calculation to calculate a center of gravity of the digital signals based on the received positional 

25 information of the corresponding row and of the corresponding column. Because the dedicated data buses are provided 
both in the column direction and in the row direction, it is possible to efficiently transfer the positional information by 
using a small number of data-transfer line systems. The positional information is required for performing the center- 
of-gravity calculation, which is a basic calculation for image processings. 

[0012] By using the dedicated data buses, it is also possible to individually access the respective processing ele- 
30 ments. 

[0013] For example, it is preferable that the control circuit includes a control portion for controlling a predetermined 
processing element to perform a predetermined calculation onto the digital signals by controlling each column-direction 
data-transfer data line and each row-direction data-transfer data line to perform data transfer operation to transfer 
predetermined calculation-control data. In this case, it is possible to control individual processing elements to perform 

35 different calculation operations. 

[0014] It is preferable that the control circuit includes a data transfer control portion for controlling each column- 
direction data-transfer data line and each row-direction data-transfer data line to perform data transfer operation to 
transfer predetermined calculation-control data, thereby allowing calculation result data, obtained at a processing el- 
ement which is located at a corresponding row and a corresponding column, to be transferred to the control circuit. In 

40 this case, it also becomes possible to transfer a calculation result obtained at some particular processing element to 
the control circuit. 

[0015] It is preferable that the high-speed vision sensor may further comprise data buffers which are provided in 
correspondence with the column-direction data-transfer bus and the row-direction data-transfer bus, respectively. By 
providing the high-speed vision sensor with the data buffers, it is possible to perform high speed data transfer operation 
even when data transfer speed between the control circuit and the data transfer buses is low. By integrating together 
the parallel processing system, the data transfer buses, and the data buffers, it is possible to relatively easily enhance 
the data transfer speed, at which data is transferred between the processing elements and the buffers. 
[001 6] It is preferable that the parallel processing system further includes a shift register array, the shift register array 
having a plurality of shift registers which are disposed in one-to-one correspondence with the plurality of analog-to- 

50 digital converters and in one-to-one correspondence with the plurality of rows of processing elements, each shift register 
successively transferring digital signals, which are outputted from the corresponding analog-to-digital converter and 
which are equivalent to signals outputted from the photodetectors in a corresponding photodetector row, to predeter- 
mined processing elements in the co responding row. The shift registers are used as being dedicated to transferring 
data to the processing elements. Accordingly, calculation processing can be performed even during the transfer proc- 

55 ess. Processing can be performed efficiently by reducing the wait time for both of the transfer process and the calculation 
process, thereby reducing the overall processing time. It is possible to achieve a pipeline operation, and to perform a 
high-speed image processing, and particularly real-time processing. 
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BRIEF DESCRIPTION OF THE DRAWINGS 
[0017] 

5 Fig. 1 is a block diagram showing a high-speed vision sensor according to an embodiment of the present invention; 

Fig. 2 is a schematic view showing the construction of the high-speed vision sensor of the embodiment; 
Fig. 3 is a structural block diagram showing a control circuit employed in the high-speed vision sensor of the 
embodiment; 

Fig. 4 is a circuit diagram showing a photodetector array and an analog-to-digital converter array provided in the 
10 high-speed vision sensor of the embodiment; 

Fig. 5 is a detailed circuit diagram showing the integrating circuit in the analog-to-digital converter array of Fig. 4; 
Fig. 6 is a block diagram showing a processing element and a shift register employed in the high-speed vision 
sensor of the embodiment; 

Fig. 7(A) is a circuit diagram showing a register matrix employed in the processing element of Fig. 6; 
15 Fig. 7(B) is a control timing chart of the processing element of Fig. 6; 

Fig. 8 is a flowchart showing the operation of the high-speed vision sensor of the embodiment; 
Fig. 9 is a flowchart of a center of gravity calculating process, which is one example of a step S11 0 in Fig. 8; 
Fig. 10(A) is a flowchart of an image intensity total sum calculation process in the center of gravity calculating 
process of Fig. 9. 

20 Fig. 1 0(B) is a flowchart of a total sum calculating process in the image intensity total sum calculation process of 

Fig. 10(A); 

Fig. 1 0(C) is a flowchart of a pixel selection process in the image intensity total sum calculation process of Fig. 1 0(A); 
Fig. 11 is a flowchart of an x direction coordinate calculating process in the center of gravity calculating process 
of Fig. 9; 

25 Fig. 1 2 is a flowchart of a y direction coordinate calculating process in the center of gravity calculating process of 

Fig. 9; 

Fig. 13 is a flowchart of a desired position calculation process, which is another example of the stepS110 in Fig. 8; 
Fig. 14 is a flowchart of a calculation-output process, which is still another example of the step S110 in Fig. 8; 
Fig. 15 is a flowchart of an image searching process, which is another example of the step S11 0 in Fig. 8; 
30 Fig. 16 is a flowchart of an edge-enhancement process, which is one example of a preprocess executed during 

the step S110 in Fig. 8; 

Fig. 1 7 is a block diagram showing a modification of the high-speed vision sensor of the present invention; and 
Fig. 1 8 is a block diagram showing another modification of the high-speed vision sensor of the present invention. 

35 BEST MODE FOR CARRYING OUT THE INVENTION 

[0018] A high-speed vision sensor according to a preferred embodiment of the present invention will be described 
while referring to Figs. 1-16. 

[0019] It is noted that the same parts or components are designated by the same reference numerals to avoid du- 
40 plbating description. 

[0020] Fig. 1 is a block diagram showing the high-speed vision sensor 1 0 according to the present embodiment. Fig. 
2 shows an example of the structure of the sensor 10. 

[0021] First, a brief description will be given for the overall configuration of the high-speed vision sensor 10, with 
reference to Fig. 1 . The high-speed vision sensor 1 0 of the present embodiment includes a photodetector array 11 , an 
45 analog-to-digital converter array 13 ; a parallel processing system 14, a control circuit 15, an x-direction data bus 17, 
a y-direction data bus 18, an x-direction data buffer 19, a y-directlon data buffer 20, an instruction/command bus 16, 
and an output bus 155 (Fig. 6) to be described later. 

[0022] N1 x N2 photodetectors 120 are arranged two-dimensionally (that is, N2 rows x N1 columns) in the photode- 
tector array 11. In other words, a horizontal photodetector row 110 is defined to include N1 photodetectors 120 which 

50 are lined up in the horizontal direction (X direction). The photodetector array 11 includes N2 horizontal photodetector 
rows 110, which are arranged vertically (Y direction), that is, orthogonal to the horizontal direction (X direction). 
[0023] The analog-to-digital converter array 1 3 includes N2 analog-to-digital converters 210 which are arranged one- 
dimensionally (vertically (y direction)). N2 analog-to-digital converters 21 0 are arranged in one-to-one correspondence 
with the N2 horizontal photodetector rows 110 in the photodetector array 11 . Each analog-to-digital converter 210 is 

55 for successively converting electric charges, outputted from photodetectors 120 that belong to a corresponding hori- 
zontal photodetector row 110, into voltage signals and then further analog-to-digital converting the voltage signals into 
digital signals. 

[0024] The parallel processing system 14 is provided with a processing element array 40. The processing element 
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array 40 includes N1 x N2 processing elements (PE) 400 which are arranged two-dimensionally (that is, N2 rows x N1 
columns) in one-to-one correspondence with the photodetectors 1 20. The processing element array 40 is also provided 
with N2 shift register lines 420 (lines of shift registers for transferring image data) in one-to-one correspondence with 
the N2 rows of processing elements. Each shift register line 420 includes N1 shift registers 410 (shift registers for 
5 image data transfers). The total number N1 of the shift registers in each line 420 is equal to the total number of process- 
ing elements 400 in the corresponding row. In each shift register line 420, the constituent N1 shift registers 410 are 
connected with one another in series. Each shift register 41 0 is connected to a corresponding processing element 400 
in the corresponding row. 

[0025] The control circuit 15 is for controlling the entire circuit in the high-speed vision sensor 10 by transmitting 
10 command signals and data thereto. The x-direction data bus 17 is connected to the processing elements 400 in the 
respective columns. The x-direction data bus 1 7 is for transferring data to each processing element 400 in each column. 
The y-direction data bus 18 is connected to the processing elements 400 in the respective rows. The y-direction data 
bus 1 8 is for transferring data to each processing element 400 in each row. The x-direction data buffer 1 9 is connected 
between the x-direction data bus 17 and the control circuit 15. The y-direction data buffer 20 is connected between 
is the y-direction data bus 18 and the control circuit 15. In this way, the x-direction data bus 17 is connected to the control 
circuit 1 5 via the x-direction data buffer 1 9. The y-direction data bus 1 8 is connected to the control circuit 1 5 via the y- 
direction data buffer 20. The instruction/command bus 16 is for transferring signals from the control circuit 15 to the 
photodetector array 11 , the analog-to-digital converter array 13, and the parallel processing system 14. As shown in 
Fig. 6 to be described later, the control circuit 15 is further connected to all the processing elements 400 via the single 
20 output bus 155. 

[0026] With this configuration, N2 data lines are used in the high-speed vision sensor 1 0 to connect the photodetector 
array 11 with the parallel processing system 14. Accordingly, the photodetector array 11 and the parallel processing 
system 14 can be formed on separate circuit boards, as shown in Fig. 2, in a configuration that enables the operations 
of each device to be verified separately. Therefore, production of the high-speed vision sensor 10 can be more stable. 

25 Further, by forming the photodetector array 1 1 and the parallel processing system 1 4 on separate circuit boards in this 
way, it is possible to manufacture both devices with a high level of integration. In addition, it is possible to employ a 
processing method suitable for the characteristics of each device, thereby enabling a more stable production. The data 
buses 1 7 and 1 8 and the data buffers 1 9 and 20 for the x- and y- directions may be formed on the same circuit board 
with the parallel processing system 14 as shown in the drawing. Alternatively, they may be formed on circuit boards 

30 which are prepared separately from the circuit board, on which the parallel processing system 14 is formed. It is es- 
pecially preferable that the data buses 1 7 and 1 8 and the data buffers 1 9 and 20 are integrated together with the parallel 
processing system 14. It becomes easy to increase the data transfer speed between the data buffers 1 9 and 20 and 
the processing elements 400. Because each component of the high-speed vision sensor 10 can be entirely created 
using the CMOS process, it is possible to create all the components on one chip, thereby greatly reducing production 

35 costs. 

[0027] Next, the components of each circuit will be described 

[0028] Fig. 3 is a block diagram showing the construction of the control circuit 15. The control circuit 15 includes a 
CPU 150, a memory 151 , an image reading control portion 300 (see Fig. 4), and an input/output interface 152. All of 
these components are connected by a bus 153. The input/output interface 152 is for performing input/output operation 

40 with an external device. The memory 1 51 stores therein data of programs to be executed by the CPU 1 50. The programs 
include: a vision sensor process program (shown in Fig. 8) which will be described later; and an image process program 
for controlling the parallel processing system 1 4 to execute a parallel image processing (Figs. 9-12, Fig. 1 3, Fig. 14, 
Fig. 15, or Fig. 16, for example) during an image processing step (S110 in Fig. 8) of the vision sensor process program. 
Data of the vision sensor process program and of the image process program is written in the memory 151 via the I/ 

45 o interface 152 by an external device 1000 (an external computer, for example). The bus 153 includes a command 
bus and a data bus (not shown), and is connected to: the instruction/command bus 16, the data buffers 19 and 20 
shown in Fig. 1, and the output bus 155 shown in Fig. 6. 

[0029] Based on the vision sensor process program stored in the memory 151 , the CPU 150 controls the photode- 
tector array 11 and the analog-to-digital converter array 13, via the image reading control portion 300, and controls the 

50 parallel processing system 14. More specifically, the CPU 150 controls the image reading control portion 300 (Fig. 4) 
(during S101 of Fig. 8) to pick up images by using the photodetector array 11 and the analog-to-digital converter array 
13. The CPU 150 also controls both of: data transfers executed by the shift registers 410 in the parallel processing 
system 14, and calculations executed by the processing elements 400 (in S102 - S104 and in S1 10 of Fig. 8), thereby 
achieving a single instruction and multi-data stream (SIMD) parallel processing. The CPU 1 50 further performs required 

55 calculations while the parallel processing system 1 4 is executing the parallel processing. The CPU 1 50 further performs 
other necessary calculations, and makes determinations based on processing results obtained by the parallel process- 
ing system 14. By using the input/output interface 152, the CPU 150 communicates with the external device 1000, 
such as an external computer, or controls another external device 1000, such as an external actuator. For example, 
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the CPU 150 outputs the calculated results to the external computer or controls the external actuator based on the 
calculated results. 

[0030] Next, the structure of the photodetector array 11 and the analog-to-digital converter array 13, which serve as 
an image reading portion, will be described in detail with reference to Figs. 4 and 5. 

5 [0031 ] ^ The photodetector array 1 1 functions as a light receiving unit for detecting light. The analog-to-digital converter 
array 13 functions as a signal processing unit for converting electric current signals outputted from the photodetector 
array 1 1 into voltage signals and further for performing ana!og-to-digital conversion onto the voltage signals. The image 
reading control portion 300, in the control circuit 15, is connected to the photodetector array 11 and the analog-to- 
digital converter array 13. The image reading control portion 300 functions as a timing control unit for transmitting 

10 instruction signals, indicative of operation timings, to the photodetector array 11 and the analog-to-digital converter 
array 13. 

[0032] First, the structure of the photodetector array 11 (light receiving portion) will be described. 

[0033] As shown in Fig. 4, each photodetector 120 is configured from a photoelectric conversion element 130 and 

a switch element 140. The photoelectric conversion element 130 is for generating an electric charge according to the 

15 intensity of received light. The switch element 1 40 is connected to a signal output terminal of the photoelectric conver- 
sion element 130. The switch element 140 outputs the electric charge, accumulated in the photoelectric conversion 
element 130, in response to a horizontal scan signal Vj (i = 1 — N1). Each horizontal photodetector row 110 is con- 
structed from N1 photodetectors 120 which are arranged in the horizontal direction (x-direction) with their switch ele- 
ments 140 being connected with one another. The photodetector array 11 is constructed from N2 horizontal photode- 

20 tector rows 110 which are arranged vertically (in the y-direction), that is, perpendicularly to the horizontal direction. 
Thus, the photodetectors 120^ (i=1 ~N1 , j=1 — N2) are arranged two-dimensionally in N1 columns x N2 rows. 
[0034] Next, the structure of the analog-to-digital converter array 13 will be described with reference to the same 
figure 4. The analog-to-digital converter array 13 serves as the signal processing unit. 

[0035] The analog-to-digital converter array 13 is constructed from N2 analog-to-digital converters 21 0j (j=1 ~ N2). 
25 Each analog-to-digital converter 21 0j is for individually extracting electric charges transmitted from the respective pho- 
todetectors in the corresponding horizontal photodetector row 11 0j (j=1 ~N2), for processing the electric charges, and 
for outputting digital signals corresponding to the magnitudes of the electric charges. 

[0036] Each analog-to-digital converter 21 0j includes: an integrating circuit 220j having a charge amp 221 j; a com- 
parator circuit 230j; and a capacity control mechanism 240j. 

30 [0037] The integrating circuit 220, includes: the charge amp 221 y a variable capacity unit 222j; and a switch element 
223j. The charge amp 221 j is for amplifying the charge of an input signal received from the horizontal photodetector 
row 110j. The variable capacity unit 222j is connected, on one end, to the input terminal of the charge amp 221 j, and 
is connected, on the other end, to the output terminal of the same. The switch element 223j is connected, on one end, 
to the input terminal of the charge amp 221 y and is connected, on the other end, to the output terminal of the same. 

35 The switch element 223j is turned ON or OFF in response to a reset signal R which is supplied from the image reading 
control portion 300, thereby toggling the operation of the integrating circuit 220j between an integrating state and a 
not-integrating state. 

[0038] Fig. 5 shows a more detailed construction of the integrating circuit 220. This drawing shows one example of 
the integrating circuit that has an analog-to-digital converting function with a 4-bit (16 gradation) resolution. The inte- 

40 grating circuit 220 will be described below for this circuit structure. 

[0039] The variable capacity unit 222 includes: capacity elements C1 - C4; switch elements SW11 - SW1 4; and other 
switch elements SW21 - SW24. Each of the capacity elements C1 - C4 is connected, on one end, to the input terminal 
of the charge amp 221 that receives signals outputted from the corresponding horizontal photodetector row 110. Each 
of the switch elements SW11 - SW14 is connected between the other end of a corresponding capacity element C1 - 

45 C4 and the output terminal of the charge amp 221 . The switch elements SW1 1 - SW1 4 are for opening and closing in 
response to capacity instructions signals C 11 - C 14 , respectively. Each of the switch elements SW21 - SW24 has one 
terminal, which is connected between the corresponding capacity element C1 -C4 and the corresponding switch element 
SW11 - SW14, and another terminal, which is connected to a ground level GND. The switch elements SW21 - SW24 
are for opening and closing in response to capacity instructions signals C 2 i - C 24 , respectively. The capacities C 1 - C 4 

so of the capacity elements C1 - C4 have the following relationships: 

C 1 =2C 2 = 4C 3 = 8C 4 



[0040] Wherein C 0 is the maximum electrical capacity required by the integrating circuit 220. It is assumed that the 
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saturation charge of the light receiving element 1 30 (see Fig. 4) is Q 0 and that a reference voltage is V REF . In this case, 
the following relationship is satisfied: 

5 c o = Qo^ref 

[0041] The remainder circuits in the analog-to-digital converter 210j other than the integrating circuit 220j will be 
described below while once again referring to Fig. 4. The comparator circuit 230j is for comparing the value of an 
integral signal V s , which is outputtedfrom the integrating circuit 220j, with the reference voltage V REF , andforoutputting 

10 a comparison result signal V c . Based on the value of the comparison result signal V c , the capacity control mechanism 
240j outputs a capacity instruction signal C to notify the variable capacity unit 222j in the integrating circuit 220j. The 
capacity control mechanism 240j also outputs a digital signal D1 that corresponds to the capacity instruction signal C. 
[0042] It is noted that the analog-to-digital converter array 13 has been described above for the case where the 
analog-to-digital converter array 1 3 has a 4-bit (1 6 gradation) resolution. However, the analog-to-digital converter array 

15 13 can be configured to have a 6-bit, 8-bit, or other bit resolution. 

[0043] The image reading unit 11 . 13, having the above-described structure, is controlled in image reading timings 
by the image reading control portion 300 in the control circuit 15. As shown in Fig. 4, the image reading control portion 
300 includes: a basic timing portion 310, a horizontal shift register 320, and a control signal portion 340. The basic 
timing portion 310 is for generating a basic timing to attain clock control of the entire circuits 11 and 13. The horizontal 

20 shift register 320 is for generating a horizontal scan signal Vj according to a horizontal scan instruction inputted from 
the basic timing portion 310. The control signal portion 340 is for generating a reset instruction signal R. 
[0044] Next, the structure of the parallel processing system 14 will be described. 

[0045] As described already, as shown in Fig. 1 , the parallel processing system 14 is provided with N2 shift register 
lines 420j (j = 1 ~ N2) in correspondence with the N2 horizontal photodetector rows 1 1 0j (j = 1 ~ N2) in the photodetector 

25 array 11 and in correspondence with the N2 analog-to-digital converters 21 0j (j = 1 — N2) in the analog-to-digital 
converter array 13. In each shift register line 420j, N1 shift registers 410,j are connected in series. Each shift register 
410jj has a plurality of bits (four (4) bits in this example). As shown in Fig. 6, the controller 15 is connected to each 
shift register 41 Ojj (1=1 — N1 , j = 1 — N2) via the instruction/command bus 16. The controller 15 controls transfer of 
data from the analog-to-digital converter array 13 to the required position by outputting a transfer start signal to each 

30 shift register 41 Oj j. 

[0046] In the parallel processing system 14, the processing elements 400j j (i = 1 — N1 , j = 1 — N2) are arranged 
two-dimensionally in one to one correspondence with the N1 x N2 photodetectors 120jj (i = 1 ~ N1 , j = 1 ~ N2). Each 
processing element 400g (i = I ~ N1, j = 1 ~ N2) is connected to a corresponding shift register 41 Og (i = 1 ~ N1 , j = 
1 - N2) in a corresponding shift register line 420j (j = 1 ~ N2). As shown in Fig. 6, the controller 15 is connected to 
35 each processing element 400g via the instruction/command bus 16, and controls calculations executed by each 
processing element 400. 

[0047] Next, the output bus 155 will be described. 

[0048] As shown in Fig. 6, all the processing elements 400 are connected to the single output bus 155. The output 
bus .155 is constructed from a single one-bit signal line. Because the output bus 155 is connected to all the processing 

40 elements 400, the output bus 155 functions as a wired OR circuit for transmitting, to the control circuit 15, a logical OR 
value of outputs from all the processing elements 400. Accordingly, the control circuit 15 can receive output signals 
from all the processing elements 400 in such a state that the output signals are united together. The control circuit 15 
can therefore receive the output signals immediately without performing scanning operation. 
[0049] Next, the x-direction data bus 1 7 and the y-direction data bus 1 8 will be described. 

45 [0050] It is noted that each of the x- and y-direction data buses 1 7 and 18 can be constructed from, for example, a 
one-directional data bus that has a function to transfer data only In a single direction from the control circuit 15 to the 
processing elements 400 and to write the data in the processing elements 400. 

[0051] As shown in Fig. 1 , the x-direction data bus 17 is constructed from a plurality of (N1 , in this example) one-bit 
data lines 170i (1 ^ i ^ N1). Each one-bit data line 170i is connected to all of the N2 processing elements 400(i,j) (1 
50 ^ j ^ N2) that are located on the corresponding processing element column i. Each one-bit data line 170i is connected 
also to the control circuit 15 via the x-direction data buffer 19. With this structure, each one-bit data line 170i transfers 
data, via the x-direction data buffer 19, from the control circuit 15 to all the processing elements 400 that are located 
on the corresponding column i. 

[0052] Similarly, the y-direction data bus 1 8 is constructed from a plurality of (N2, in this example) one-bit data lines 
55 1 80j (1 g j ^ N2). Each one-bit data line 180j is connected to all the N1 processing elements 400(i,j) (1 ^ i ^ N1) that 
are located on the corresponding processing element rowj. Each one-bit data line 1 80j is connected also to the control 
circuit 15 via the y-direction data buffer 20. With this structure, each one-bit data line 180j transfers data, via the y- 
direction data buffer 20, from the control circuit 15 to all the processing elements 400 that are located on the corre- 
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sponding row i. 

[0053] Next, the structure of the processing elements 400 will be described in more detail with reference to the block 
diagram shown in Fig. 6. 

[0054] The processing elements 400 are configured to perform SIMD-type parallel processing. In this type of parallel 
5 processing, ail of the elements are controlled by common control signals. Accordingly, the number of transistors mount- 
ed in each element can be reduced, enabling higher integration in the parallel processing system 14 and enabling an 
increased number of elements. 

[0055] More specifically, each processing element 400 includes a register matrix 401 , an A-latch 402, a B-latch 403, 
and an arithmetic logic unit (ALU) 404. The register matrix 401 has 4x8 bits, and can be randomly accessed. The 

10 register matrix 401 is of a 1 -bit shift type. The register matrix 401 is for performing both of data storage operation and 
data input/output operation. More specifically, the register matrix 401 in each processing element 400 is for receiving, 
from the corresponding shift register 41 0, a digital signal D1 that is equivalent to a signal outputted from a corresponding 
photodetector 120, and for storing it therein. The register matrix 401 in each processing element 400 is directly con- 
nected to the register matrices 401 in four adjacent processing elements 400 that are disposed adjacent to the subject 

is processing element 400. The register matrix 401 can therefore receive digital signals that are stored in the register 
matrices 401 of the four neighboring processing elements and can store them therein. The register matrix 401 is con- 
nected further to a corresponding x-direction data line 170i in the x-direction data bus 17 and to a corresponding y- 
direction data line 180J in the y-direction data bus 18. The register matrix 401 can perform data transfer operation by 
using both of the data buses. The ALU 404 is for performing bit serial calculations to perform calculation one bit at a 

20 time beginning from the least significant bit. The A-latch 402 and the B-latch 403 are for receiving signals that are 
stored in the register matrix 401 , and for supplying these signals to the ALU 404 for calculations. 
[0056] In each processing element 400 having the above-described structure, the A-latch 402 and the B-latch 403 
can read data from desired registers in the register matrix 401 . The ALU 404 performs calculations based on the thus 
read data. The results of the calculations are written back to desired registers in the register matrix 401 . The processing 

25 element 400 executes various calculations by repeatedly performing this operation cycle. 

[0057] More specifically, the ALU 404 is a one-bit calculator that has functions for calculating a logical product (AND), 
a logical sum (OR), an exclusive or (XOR), an addition (ADD), a Carry-addition, and the like. Because the ALU 404 
can process only one bit at a time, the ALU 404 can execute multi-bit calculations by performing a series of one-bit 
calculations. It is noted that complex calculations can be described by a combination of the above-described plural 

30 calculations. Accordingly, the ALU 404 performs complex calculations by repeatedly performing calculations each time 
selecting one function from these calculating functions available to be performed by the ALU 404. For example, mul- 
tiplication can be described as a combination of additions. Therefore, the ALU 404 can perform multiplication by re- 
peating adding functions a plurality of times. Subtraction can be implemented by first reversing the bits in the number 
to be subtracted, adding one (1) to make the value negative, and then performing adding operation. Division can be 

35 implemented by setting the divisor to a number, such as 8, 4, or 2, and shifting bits. (For example, the bits are shifted 
three places to the right if the divisor is 8.) An absolute value for a negative number can be calculated by reversing the 
sign of the negative number, whose sign bit is 1 , that is, by reversing the bit and then adding one (1) to the result. 
[0058] As shown in Fig. 7(A), in the register matrix 401, there are arranged: 24 one-bit registers 4010 and eight 
function registers 4012. The 24 one-bit registers 4010 can be randomly accessed. The eight function registers 4012 

40 are for enabling input/output operations with external devices and with neighboring processing elements 400. All of 
these registers are treated as defining a single address space. The numbers appearing in the registers 401 0 and 4012 
in Fig. 6 indicate the addresses assigned to the corresponding registers. More specifically, the 24 registers 4010 are 
assigned addresses 0 - 23, while the eight function registers 4012 are assigned addresses 24 - 31 . With this configu- 
ration, input/output data can be accessed in the same manner as reading data from and writing data into the registers. 

45 [0059] As shown in Fig. 7(A), the register matrix 401 further includes one OR gate 4014, which is connected to all 
of these 32 registers 4010, 4012. AH of the registers 4010 and 4012 are connected to the A-latch 402 and the B-latch 
403 via the single OR gate 4014. When one address in the registers 4010 and 4012 is selected, only the selected 
register outputs its content, while all the remaining non-selected registers output zero (0). The result of the OR function 
on the values from all the registers is outputted to the A-latch 402 or the B-latch 403 as the output of the entire register 

50 matrix 401 . 

[0060] The function registers 401 2 are mainly used for performing input and output operations. More specifically, the 
function register 4012 at address 29 is connected to a corresponding shift register 410, and is used for inputting data 
from the shift register 410. The function register 4012 at address 24 is used for outputting data to register matrices 
401 of four neighboring processing elements 400 that are located on all the four sides of up, down, left, and right, 
55 respectively, of the subject processing element 400. The function register 4012 at each of the addresses 24 - 27 is 
used for inputting data from a register matrix 401 of a corresponding one of the four neighboring processing elements 
400. The function register 4012 at address 28 is connected to the output bus 155, and is used as an output address 
to the control circuit 15. It is noted that the addresses 28 of the register matrices 401 in all the processing elements 
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400(ij) are connected to the single output bus 1 55. The function register 4012 at address 30 is connected to a corre- 
sponding y-direction data line 180j in the y-direction data bus 18, and is used for inputting data from the y-direction 
data line 180j. The function register 4012 at address 31 is connected to a corresponding x-direction data line 170i in 
the x-direction data bus 1 7, and is used for inputting data from the x-direction data line 1 70L It is noted that the function 

5 register 401 2 at address 28 is configured to always read zero (0) when it is used to read data. 

[0061] The control circuit 1 5 controls accesses to the register matrix 401 and calculations by the ALU 404, thereby 
controlling the entire processes of calculations and input/output operations of the processing element 400. In order to 
write data, inputted from the shift register 41 0, to address zero (0) in the register matrix 401 , for example, the control 
circuit 15 outputs, to the processing element 400, an instruction to calculate the logical OR of 0 (register address 28) 

10 and the sensor input (register address 29) and then to write the calculated result to register address (0). It is noted that 
as shown in Fig. 7(B), the control circuit 1 5 sets one cycle of operation to the value four times as long as a basic clock 
CLK. While providing various clocks CLK2, CLK4, the control circuit 1 5 successively performs control operation to read 
data into A-latch, to read data into B-latch, and to write the result of calculation into the register matrix. By performing 
this process repeatedly over a plurality of cycles, it is possible to attain required inputting/outputting operation with the 

15 register matrix 401 and to attain calculations at the ALU 404. 

[0062] Next, the operations of the present embodiment will be described with reference to Figs. 4 - 6. 
[0063] First, the operations of the image reading portion 11 , 13 will be described. 

[0064] First, the image reading control portion 300 sets the reset signal R to enabled, and sets all of the switches 
SW11 - SW14 in the variable capacity unit 222 of Fig. 5 to ON and sets all the switches SW21 - SW24 to OFF. With 
20 this operation, the capacity between the input and output terminals of the charge amp 221 is set to C 0 . At the same 
time, all the switch elements 140 shown in Fig. 4 are set to OFF, and the horizontal scan signal Vj is set to a state that 
does not select any photodetectors 120^. From this state, the reset signal R is set to disabled, and each integrating 
circuit 220 is controlled to start integration operations. 

[0065] After starting the integration operation a horizontal scan signal V 1 is outputted for setting, to ON, only the 
25 switch element 1 40 in the first photodetector 1 20«, j among the photodetectors in each of the N2 horizontal photodetector 

rows 1 1 0j shown in Fig. 4. When the switch element is thus turned ON, a charge Q-,, which has been accumulated in 

the photoelectric conversion element 130 in response to light received until present, is outputted as a current signal 

from the photodetector array 11 . Thus, a signal is read from the photoelectric conversion element. The charge Q 1 then 

flows to the variable capacity unit 222, which is now being set at capacity C 0 . 
30 [0066] Next, the internal operations of the integrating circuit 220 will be described with reference to Fig. 5. The 

capacity control mechanism 240 (Fig. 4) opens switches SW12 - SW14, and then closes the switches SW22 - SW24. 

As a result, the integral signal V s is outputted to have the following voltage value: 



35 



45 



55 



V s = Q/C 1 



[0067] The integral signal V s is inputted into the comparator circuit 230, and is compared with the reference voltage 
V REF . If the difference between the integral signal V s and the reference voltage V REF is within the range of resolution, 
that is, lower than or equal to ±C 4 /2 : then the integral signal V s is determined to match the reference voltage V REF . In 
40 this case, no further capacity control is performed and the integral operation ends. On the other hand, if the difference 
does not fall within the range of resolution, then further capacity control will be performed to continue the integral 
operation. For example, if V s is greater than V REF , the capacity control mechanism 240 opens the switch SW22 and 
subsequently closes the switch SW1 2. As a result, the integral signal V s changes to have the following voltage value: 



V S = Q/(C 1 +C 2 ) 



[0068] This integral signal V s is inputted into the comparator circuit 230 (Fig. 4) and is compared with the reference 
voltage V REF . On the other hand, if V s is smaller than V REF , then the capacity control mechanism 240 opens both 
so switches SW11 and SW22 and subsequently closes the switches SW12 and SW21 . As a result, the integral signal V s 
changes to have the following voltage value: 



V s = Q/C 2 

[0069] This integral signal V s is outputted to the comparator circuit 230 to be compared with the reference voltage 
Vref- 

[0070] By repeating this feedback loop among the integrating circuit 220, the comparator circuit 230, the capacity 



BNSDCCID: <EP 1 164544A1_I_> 



EP1 164 544 A1 



control mechanism 240, and back to the integrating circuit 220, comparison operation and capacity setting operation 
(that is, control of the ON/OFF settings of the switches SW11 - SW14 and SW21 - SW24) are repeated until the integral 
signal V s matches the reference voltage V REF within the resolution range. The values of the capacity instructions 
signals - C 14; which indicate the ON/OFF states of the switches SW11 - SW14 at the time when the integration 

5 operations are ended, constitute a digital signal that corresponds to the charge Q 1f wherein the most significant bit 
(MSB) of the digital signal is the value of , while the least significant bit (LSB) is the value of C 14 . The input signal 
is thus analog-to-digital converted into the digital signal, and the digital signal is outputted as a digital signal D1 to the 
processing element array 14. As described above, the present apparatus successively determines the values of the 
respective bits in the digital signal D1 one bit at a time from the MSB to the LSB. 

10 [0071 ] Hence, according to the present embodiment, the integral signal V s is repeatedly compared with the reference 
voltage V REF , while the capacity elements C1 - C4 are successively turned ON one at a time. The comparison result 
is outputted as a digital signal D1 . More specifically, C1 is first set to ON, thereby making integral signal V s equal to 
Q/C1 . This integral signal V s is compared with the reference voltage V REF . A one (1) is outputted as the MSB (most 
significant bit) if the integral signal V s is larger than the reference voltage V REF . A zero (0) is outputted as the MSB 

15 (most significant bit) if the integral signal V s is smaller than the reference voltage V REF . Next, C2 is set to ON, making 
the integral signal V s be equal to Q/(C1 + C2) when MSB = 1 , or the integral signal V s be equal to Q/C2 when MSB 
= 0. This integral signal V s is compared to the reference voltage V REF . A one (1) is outputted as the second bit if the 
integral signal V s is larger than the reference voltage V REF . A zero (0) is outputted as the second bit if the integral 
signal V s is smaller than the reference voltage V REF . Analog-to-digital conversion is executed by repeating the above- 

20 described process the number of times that is equal to the number of bits required. 

[0072] When output of a digital signal equivalent to the photoelectric output from the first photodetector 120-,j is 
completed, the reset signal R is enabled. The reset signal is then disabled again, and the capacity value of the variable 
capacity unit 222j is initialized. Subsequently, a horizontal scan signal V 2 is outputted for turning ON the switch element 
140 only in the second photodetector 120 2 j in each row 11 Oj. Then, the above-described process is performed again 

25 for reading the photoelectric output from the second photodetector 120 2 j and for outputting a corresponding digital 
signal. Hereafter, by successively changing the horizontal scan signal V h photoelectric outputs are read successively 
from all the photodetectors 120, and corresponding digital signals are outputted to the parallel processing system 14. 
[0073] Next, the operations of the processing elements 400 will be described with reference to Fig. 6. 
[0074] The digital signal of the output from each photodetector 120jj, which is produced by the analog-to-digital 

30 conversion, is transmitted to the register matrix 401 of a corresponding processing element 400^ via the shift registers 
41 0. This transmission process is executed, in the corresponding shift register line 420, by transferring the digital signal 
successively from the shift register 41 0 at one pixel to the shift register 41 0 at the next pixel . Because the shift registers 
410 are provided in the present embodiment, signal transmission operation by the shift registers can be performed 
independently from the calculation processing performed by the processing elements 400. Accordingly, it is possible 

35 to perform a pipeline process to control the shift registers 410 to transmit next data while controlling the processing 
elements 400 to perform calculation processing onto present data. It is therefore possible to perform calculations at 
the processing elements 400 at a higher frame rate. It is noted that the shift registers 41 0 begin transferring the analog- 
to-digital converted data in response to a transfer start signal, which is sent from the control circuit 1 5. The shift registers 
410 return a "data transfer complete" signal to the control circuit 15 after performing bit-shifting transfer operation by 

^0 the number of bits that is equal to the product of the total number of elements in a row and the number of bits of analog 
data levels. Thus, an efficient transmission operation can be attained. Hence, according to the present embodiment, 
the pipeline process can be executed by performing the calculation processes and the transmission processes in 
parallel, it is possible to reduce the amount of wait time between the calculation processes and the transmission proc- 
esses in successive frames, thereby achieving faster image processing. 

45 [0075] The image process calculations are conducted in the processing elements 400 as described below. Signals 
stored in the register matrices 401 of the respective processing elements 400 are transmitted to other processing 
elements 400 when required. Data and control signals are transferred from the control circuit 15 to the processing 
elements 400 via the x-direction data bus 17, the y-direction data bus 18, and the buffers 19 and 20. Thereafter, in 
each processing element, signals for calculations are read from the register matrix 401 into the A-latch 402 and the 

so B-latch 403. Predetermined calculation is achieved in the ALU 404. The calculated result is outputted via the register 
matrix 401 and the output bus 155 to the control circuit 15. 

[0076] It is noted that in the parallel processing system 14, the above-described image processing calculations are 
performed simultaneously by all the processing elements 400 in a parallel processing manner. Accordingly, calculation 
can be performed at an extremely high rate of speed. The control circuit 15 outputs the results of calculations, which 
55 are obtained by the parallel processing system 14, to the external device 1000 such as an external computer or other 
external equipments. For example, the results of calculations obtained by the parallel processing system 14 can be 
used as ON/OFF signals for an external equipment. The control circuit 15 may perform required calculations based 
on the results of calculations obtained by the parallel processing system 14, and then output the calculated results to 
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the external circuit 1000. 

[0077] Next, a series of vision sensor processes performed by the present vision sensor 10 will be described with 
reference to Fig. 8. This series of processes covers the image input process and the image transmission process up 
through the completion of the calculation process. 
5 [0078] The CPU 150, in the control circuit 15, controls in S101 the image reading control portion 300 (see Fig. 4), 
thereby successively switching the reset signal R between the enabled condition and the disabled condition and suc- 
cessively switching the horizontal scan signal Vj. As a result, in each row j (j = 1 , N2) of the photodetector array 11 , 
image data (frame data; hereinafter referred to as l(x, y)), which is outputted from the photodetectors 120jj (hereinafter 
referred to as photodetectors 120 (x, y)), is successively inputted into the parallel processing system 14 via the corre- 
10 sponding analog-to-digital converter 21 0j. 

[0079] In S1 02, in each row j, data inputted from the analog-to-digital converter 21 0j is transmitted in succession via 
the corresponding shift registers 41 0. This transfer process continues in S1 03 until image data from each photodetector 
120 reaches the shift register 410jj at the corresponding position (i, j) (hereinafter referred to as (x, y)). 
[0080] Aftertransmission has been completed, in S1 04, data l(x, y) of each pixel is transferred from the corresponding 

is shift register 410 to the register matrix 401 of the corresponding processing element 400jj (hereinafter referred to as 
processing element 400(x, y)). As shown in Fig. 6, data in the shift register 410 is composed of a plurality of bits (four 
bits in this example). Accordingly, data is transferred from the shift register 41 0 one bit at a time into the register matrix 
401. In S110, each processing element 400 is controlled to perform necessary image processing. 
[0081 ] When each shift register 41 0 completes transferring data to the corresponding processing element 400, while 

20 each processing element 400 is controlled to perform calculation processes in S 1 1 0, each shift register 41 0 continues 
to the process of the next frame in S1 05, and the processes of S1 01 - S103 are executed again. That is, the photode- 
tector array 11 , the analog-to-digital converter array 13, and the transfer shift registers 410 are controlled to perform 
input/transfer operation for the next frame. With respect to the parallel processing performed by the processing ele- 
ments 400, after the processing for one frame (S1 1 0) is completed, each processing element 400 advances to the next 

25 frame process in S106. As a result, image data for the next frame is transmitted in S1 04 from each shift register 410 
to the register matrix 401 of the corresponding processing element 400. Then, the parallel processing for the next 
frame is started in S110. By repeating the above-described processes, while the photodetector array 11 , the analog- 
to-digital converter array 13, and the transfer shift registers 41 0 input and transfer data for the next frame in S101 - 
S103, the processing elements 400 can perform the calculation process onto the present frame data in S11 0, thereby 

30 reducing wasteful wait time. 

[0082] Next, the parallel image processing operations of the processing elements 400 executed in S110 will be 
described in detail with reference to several examples of the image processings. During the image processing proce- 
dure, the CPU 150 in the control circuit 15 performs parallel-control onto all the processing elements 400 simultane- 
ously, thereby achieving calculation processings very rapidly. 

35 

(Example 1) 

[0083] According to the present embodiment, calculation data and calculation-control data can be supplied to each 
processing element within a short period of time via the x-direction data bus 1 7 and the y-direction data bus 1 8 from 

40 outside. Accordingly, calculation can be attained at a high speed even when each processing element 400 performs 
in S11 0 a type of calculation that requires additional data other than image information. For example, in order to calculate 
the center of gravity of the input image during the image processing step (S110), the processing elements require 
positional information data. Accordingly, positional data is supplied from outside to the processing elements, which in 
turn calculate the center of gravity. Next will be described in detail the operation performed when the center of gravity 

45 is calculated during the image processing procedure (S1 1 0). 

[0084] It is noted that the memory 151 in the control circuit 15 previously stores therein x-direction positional infor- 
mation (which will be referred to simply as "x address" hereinafter) and y-direction positional information (which will 
be referred to simply as "y address" hereinafter) of each pixel. In more concrete terms, the memory 151 stores, as x- 
address, binary number data indicative of "1" - "N1", and stores, as y-address : binary number data indicative of "1" - 

so »N2". 

[0085] The processing for calculating the center of gravity is based on a calculation of a weighted sum in each of x- 
and y-directions. The center of gravity Gc of an image is determined by calculating the following formula (1 ) with respect 
to image intensity data l(x,y) at each pixel position (x,y) of the subject image: 

55 
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(1) 



10 



[0086] That is, in order to calculate the center of gravity, each processing element 400 has to multiply its image data 
l(x,y) by its positional information (x,y). The multiplied results obtained on the plural processing elements 400 are added 
together. Accordingly, when it is desired to calculate the center of gravity, pixel positional information (x,y) is transferred, 
in advance, to each processing element. 

15 [0087] In order to calculate an x-coordinate of the center of gravity of an image, an x-directional moment "x • l(x,y)" 
is first calculated by multiplying image intensity l(x,y) with x-directional positional information. Then, an x-directional 
moment sum "Zlx • l(x,y)" is calculated by adding together the x-directional moments of all the pixels. The x-coordinate 
of the center of gravity is calculated by dividing the x-directional moment sum "£ £x • l(x,y)" by an image intensity sum 
"2 Zl(x,y)\ It is noted that the image intensity sum T X l(x,y)" is calculated by adding image intensities of all the pixels. 

20 Similarly, in order to determine a y-coordinate of the center of gravity of the image, a y-directional moment "y • l(x,y)" 
is first calculated at each pixel by multiplying the image intensity l(x,y) by the y-directional positional information. Then, 
a y-directional moment sum "IZy» l(x,y)" is calculated by adding together the y-directional moments of all the pixels. 
The y-coordinate of the center of gravity is calculated by dividing the y-directional moment sum " I Zy • I (x,y) " by the 
image intensity sum " Z £ I (x. y) H . 

25 [0088] Accordingly, in order to determine the position of the center of gravity of an input image, it is preferable to 
execute the center of gravity calculation processes (S300), shown in Fig. 9, during the image processing step S110 



[0089] During the center of gravity calculation processes (S300), first, the image intensity sum £ X l(x,y) is calculated 
in S350. Next, in S400, the x-coordinate of the center of gravity is calculated. Thereafter, in S500, the y-coordinate of 
30 the center of gravity is calculated. 

[0090] It is noted that when the calculation process for the center of gravity (S300) is started, image data l(x,y) at 
each pixel of an input image D has already been transferred in S104 (Fig. 8) to the register matrix 401 of the corre- 
sponding pixel 400(x,y) via the corresponding transfer shift register 410, and is being stored in some region of the 
register matrix 410. 

35 [0091] First, the process (S350) of calculating the image intensity sum X 2 l(x,y) will be described with reference to 
Fig. 10(A). 

[0092] During the image intensity sum calculation process (S350), the CPU 150 first performs in S360 a sum calcu- 
lation process for determining the total sum of the image intensities l(x,y). 
[0093] The sum calculation process (S360) will be described with reference to Fig. 1 0(B). 

40 [0094] First, in S1 002, objective data (in this example, image data l(x,y), for which a sum is desired to be determined) 
is read from the register matrix 401 to the A latch 402. That is, data l(x,y) is transferred from its least significant bit 
from the register matrix 401 to the A latch 402. Next, in S1 004, data transfer is performed between neighboring register 
matrices 401 so that each processing element 400(x,y) receives objective data (in this example, l(x+1 , y)) from another 
processing element 400(x+1 , y) that is located at a pixel position (x+1 , y) adjacent to the subject pixel location (x,y) in 

45 the x direction. The processing element 400(x,y) stores therein the received objective data. More specifically, by using 
the transfer function of the register matrix of each pixel with its neighboring register matrix, an image data value is 
transferred to a vacant region in the register matrix of each pixel from the register matrix of the neighboring pixel. Next, 
in S1006, the objective data (image data l(x+1 , y)), which has been transferred from the neighboring pixel (x+1 , y), is 
read into the B latch 403. In S1 008, data in A latch 402 and data in B latch 403 are added together. The added result 

so is stored in the A latch 402 via the register matrix 401 in S1010 and in S1012. In this way, each pixel 400(x,y) stores 
therein a sum of the objective data of two adjacent pixels (in this example, a sum of image data l(x,y) and l(x+1 , y)). 
[0095] Next, in S1014, the CPU 15 initially sets one (1) to a variable i, which will be used to calculate a sum in the 
x direction. In S1018, transfer between neighboring register matrices in x direction is performed twice so that each 
processing element 400(x,y) receives a calculated result (in this case, l(x+2, y) + l(x+3, y)) that is stored in another 

55 processing element 400(x+2, y) that is located at a position (x+n, y) (where n = 2 1 : in this case, n = 2 1 = 2) second 
closest to the subject pixel position in the x direction. The processing element 400(x,y) stores therein the received 
calculated result. Next, the calculated result data, which has been transferred from the second closest pixel (x+2, y), 
is read into the B latch 403 in S1020. Data in the A latch 402 and the data in the B latch 403 are added together in 



(Fig. 8). 
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S1022. The added result is stored in the A latch 402 via the register matrix 401 in S1024 and in S1026. As a result, 
each pixel 400(x,y) stores a sum of image data l(x,y), l(x+1 , y), l(x+2, y), and l(x+3, y) of four neighboring pixels. 
[0096] Next, i is incremented by one (1) in S1028. It is judged in S1030 whether the obtained value n = 2« exceeds 
a half N1/2 of the total number of pixels in each row. When the obtained value n = 2 1 does not exceed the value N1/2 

5 (no in S1030), the program returns to S1018. In S1018, transfer between neighboring register matrices in x direction 
is performed n times (in this example, four times) so that each processing element 400(x,y) receives a calculated result 
(image data-added result) that is stored in another processing element 400 that is located at a position (x+n, y) (where 
n = 2', in this case, n = 2 2 = 4) that is n-th closest to the subject pixel position in the x direction. The transferred data 
is read into the B latch 403 in S1020. Adding operation the same as described above is performed in S1022 - S1026. 

10 [0097] Next, while successively incrementing i by one (S1028), the processings of S1018 - S1030 are repeated. 
Thus, addition is repeated through the eighth (n = 2 3 ) neighboring pixel, the sixteenth (n = 2 4 ) neighboring pixel, the 
32-th (n = 2 5 ) neighboring pixel, the 64-th (n = 2 6 ) neighboring pixel, ... up to the (N1/2)-th neighboring pixel. As a 
result, in S1026, the added result 1(1 ,y) + l(2,y) + ... + l(N1 ,y) is obtained in the A latch 402 of a processing element 
400(1 ,y) that is located at a leading edge of the subject row. When n exceeds the value N1/2 (yes in S1030), the 

15 program proceeds to S1 032. 

[0098] During the processes downstream from SI 032, the image intensity sums, which are obtained at the respective 
rows, are added together in a similar manner as described above. As a result, the image intensity sum Z Z l(x,y), which 
is a denominator of the formula (1), is obtained. 

[0099] More specifically, j is initially set in S1 032 to zero (0). In S1 034, transfer between neighboring register matrices 
20 in y direction is performed so that each processing element 400(x,y) receives an added result that is stored in another 
processing element 400(x, y+m) that is located at a position (x, y+m) (where m = 2i, in this case, n = 2i = 1) adjacent 
to the subject pixel position in the y direction. The processing element 400(x,y) stores the received added result. Next, 
the calculated result data, which has been transferred from the adjacent pixel (x, y+1), is read into the B latch 403 in 
S1 036. Data in the A latch 402 and the data in the B latch 403 are added together in S1 038. The added result is stored 
25 jn the register matrix 401 in S1 040. As a result, the processing element 400(1 ,y) at a leading edge of each row stores 
a sum of results calculated at two rows. 

[0100] Next, j is incremented by one (1 ) in S1 044. It is judged in S1046 whether the obtained value m = 2i exceeds 
a half N2/2 of the total number of pixels in each column. When the obtained value m = 2i does not exceed the value 
N2/2 (no in S1046), the added result obtained in S 1038 is transferred from the register matrix 401 to the A latch 402 

30 in S1 048. Thereafter, the program returns to S1 034. 

[0101] Next, in S1034, transfer between neighboring register matrices in y direction is performed twice so that each 
processing element 400(x,y) receives an added result that is stored in another processing element 400 that is located 
at a position (x, y+m) (where m = 2i, in this case, position (x, y+2)) that is second closest to the subject pixel position 
in the y direction. The transferred data is read into the B latch 403 in Si 036. Adding operation the same as described 

35 above is performed in S1038 - S1 040. 

[0102] Next, while successively incrementing j by one (S1044), the processings of S1046 - S1040 are repeated. 
Thus, addition is repeated through the fourth (m = 2 2 ) neighboring pixel, eighth (m = 2 3 ) neighboring pixel, the sixteenth 
(m = 2 4 ) neighboring pixel, the 32-th (m = 2 5 ) neighboring pixel, the 64-th (m = 2 6 ) neighboring pixel, ... up to the (N2/2)- 
th neighboring pixel. As a result, in S1040, the total sum £ £ l(x,y) of the image intensities of all the pixels is obtained 

40 in the register matrix 401 of the processing element 400(1 ,1) at the leading end of the processing element array 40. 
When m exceeds the value N2/2 (yes in S1 046), the total sum calculation process is ended. 

[0103] When the total sum calculation process (S360) is completed, as shown in Fig. 10(A), the program proceeds 
to a pixel selection process (S370) to select an objective pixel position (x1 , y1 ) (the leading end pixel position (1 , 1) in 
this case). 

45 [0104] Next, the pixel selection process (S370) will be described with reference to Fig. 10(C). 

[0105] During the pixel selection process, first, in S1102, the CPU 150 transfers data (1) to an x-direction data line 
170i (which will be referred to as M x-direction data line 170x" hereinafter), among the x-direction data bus 17, that 
corresponds to the x address x1 of the objective pixel position (x1 , y1) to be selected. The CPU 150 transfers data (0) 
to all the remaining x-direction data lines 170x. In this case, the CPU 150 transfers data (1) to an x-direction data line 

50 1 70x(x = x1 =1) that corresponds tothex address x1 of the leading end pixel position (x1 (=1),y1 (= 1)) to be selected. 
The CPU 150 transfers data (0) to all the other remaining x-direction data lines 170x (x* x1). 

[0106] Next, in S11 04, each processing element 400(x,y) is controlled to store, into its register matrix 401 , data that 
is transferred from the corresponding x direction data line 170x. As a result, among all the processing elements 400, 
the register matrices 401 at the processing elements 400(1 ,y) (1 ^ y ^ N2), whose x addresses are x1 (=1 ), are stored 
55 with data (1). The register matrices in the other remaining processing elements 400 are stored with data (0). Next, the 
thus stored data is transferred to A latch 402 in S1106. 

[0107] Next, in S1108, the CPU 150 transfers data (1) to a y-direction data line 180j (which will be referred to as M y- 
direction data line 1 80y" hereinafter), among the y-direction data bus 1 8, that corresponds to the y address y1 of the 
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objective pixel position (x1 , y1 ). The CPU 1 50 transfers data (0) to all the other remaining y-direction data lines 1 80y. 
In this case, the CPU 1 50 transfers data (1 ) to a y-direction data line 1 80y (y = y1 = 1 ) that corresponds to the y address 
y1 of the leading end pixel position (x1 (= 1 ), y1 (= 1 )) to be selected. The CPU 1 50 transfers data (0) to all the other 
remaining y-direction data lines 1 80y (y * y1 ). 
5 [0108] Next, in S1110, each processing element 400(x,y) is controlled to store, into its register matrix 401, data 
transferred from the corresponding y direction data line 1 80y. As a result, among all the processing elements 400, the 
register matrices 401 at the processing elements 400(x,1 ) (1 ^ x ^N1 ), whose y addresses are yl (=1), are stored with 
data (1). The register matrices in the other remaining processing elements 400 are stored with data (0). Next, the thus 
stored data is transferred to B latch 403 in S1112. 
10 [0109] Next, in S1114, each processing element 400(x,y) is controlled to multiply the value in A latch 402 and the 
value in B latch 403 at the ALU 404. The calculated result is stored into the register matrix 401 in S1 1 1 6. As a result, 
a multiplied result (1 ) is set only at the processing element 400(x1 , y1 ) (in this case, the leading end processing element 
400(1 ,1 )) at the objective address (x1 , y1). A multiplied result (0) is set at all the other remaining processing elements 
400. In this way, the pixel selecting process (S370) is completed. 

*5 [0110] When the pixel selecting process (S370) is completed, the program proceeds to S372 (Fig. 10(A)). In S372, 
the multiplied result obtained in the pixel selecting process (S370) is transferred from the register matrix 401 to the A 
latch 402. Next, in S374, the calculated result finally obtained in the total sum calculation process (S360) is transferred 
from the register matrix 401 to the B latch 403. Next, the ALU 404 multiplies the value in A latch 402 and the value in 
B latch 403 in S376. As a result, at the processing element 400 of the leading end address (1 , 1), an image intensity 

20 total sum 2 £ l(x,y) is obtained as a multiplied result of the calculated total sum £ 2 l(x,y) and the value (1). On the 
other hand, at the other remaining processing elements 400, a zero value (0) is obtained as a multiplied result of the 
calculated total sum and the value (0). In S378, the multiplied result is stored in the register matrix 401. In S380, the 
multiplied result is outputted from the output address "28" of the register matrix 401 to the output bus 1 55. As a result, 
the multiplied results, which are obtained in S376 at all the processing elements, are outputted to the single output bus 

25 155. in this case, only the processing element 400(1 , 1) at the leading end position outputs the image intensity sum 2 
ri(x,y) as the multiplied result, while the other remaining processing elements output the zero value (0) as the multiplied 
results. Accordingly, the output bus 155 transfers, to the control circuit 15, the image-intensity sum SI l(x,y), which is 
the logical OR value for the results calculated at ail the processing elements 400. The CPU 150 stores the received 
image intensity sum £ £ I (x,y) in the memory 151 . 

30 [0111] Next, the center of gravity x-coordinate calculation processings (S400) will be described with reference to 
Fig. 11. 

[0112] First, in S402, the CPU 150 reads, from the memory 151 , x-direction positional information (x address data 
"x") of each pixel. This positional information is required to calculate the center of gravity. The CPU 150 transfers each 
x-direction positional information to the corresponding processing element 400. This transfer operation is performed 

35 by temporarily storing each x address data "x" in the x direction data buffer 19, and then transferring the x address 
data via a corresponding data line 1 70x in the x direction data bus 1 7. The x address data is transferred successively 
from its least significant bit one bit at a time in a bit serial manner. In this case, the same x address data "x" is assigned 
for all the pixels that are connected to one x direction data line 1 70x. The x address data is data of a binary number 
that is indicative of either one of values of 1 to N1 . Accordingly, each data line 170x transfers the log 2 (N1) number of 

40 bits that constitute the binary number data indicative of the corresponding x address V. Each data line 1 70x transfers 
the log 2 (N1) number of bits one bit at a time from the least significant bit. Thus, each data line 1 70x can transfer the 
x address data to all the processing elements 400(x,y) that are located at the corresponding x address. 
[0113] In each processing element 400(x,y), the transferred address data "x" is stored in the register matrix 401 in 
S404. Thereafter, the address data "x" is read into the B latch 403 from its least significant bit in S406. Image data I 

45 ( X ,y) is again read from the register matrix 401 into the A latch 402 in S408. The ALU 404 multiplies data in A latch 
402 and data in B latch 403 in S410. The multiplied result, that is, the x direction moment value x • l(x,y), is stored in 
the register matrix in S412. 

[01 1 4] Next, in S420, a total sum calculation process is executed to determine a total sum of the x direction moments 
x • I (x,y) of all the pixels. 

50 [01 1 5] During the total sum calculation process (S420), the sum calculation process, which is already described with 
reference to Fig. 1 0(B), is executed by setting the x direction moment value x • l(x,y) as objective data. 
[0116] More specifically, in S1 002, at each processing element, objective data (an x direction moment value x • l(x, 
y), for which a sum is desired to be determined) is transferred from the register matrix 401 to the A latch 402. Next, in 
S1004, each processing element 400(x,y) receives objective data (in this example, (x+1) ♦ l(x+1, y)) from another 

55 processing element 400(x+1 , y) that is located at an adjacent pixel position (x+1 , y) in the x direction. Next, in S1006, 
the x-direction moment value (x+1) ♦ I (x+1 , y) is stored into the B latch. In S1008 - S1012, by adding data in A latch 
402 and data in B latch 403, x direction moment values in the adjacent two pixels are added together, and the added 
result is stored in the A latch 402 via the register matrix 401 . In this way, each pixel 400(x,y) stores therein a sum (x ♦ 
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l(x,y) + (x+1 ) • l(x+1 , y)) of x direction moment data at two adjacent pixels. 

[0117] Next, i is initially set in S1014 to one (1). In S1018 - S1026, transfer/adding process is performed so that the 
calculated result (x ♦ l(x,y) + (x+1 ) • l(x+1 , y)) at each processing element is added with another calculated result ((x+2) 
• l(x+2,y) + (x+3) • l(x+3, y)) that is stored in another processing element 400(x4-2, y) that is located at a position (x+n, 

5 y) (where n = 2', in this case, n = 2 1 = 2) second closest to the subject pixel position in the x direction. 

[0118] Next, while successively incrementing i by one (S1028), the processings of S1018 - S1030 are repeated. 
Thus, transfer/addition operation is repeated through the fourth (n = 2 2 ) neighboring pixel, the eighth (n = 2 3 ) neighboring 
pixel, the sixteenth (n = 2 4 ) neighboring pixel, the 32-th (n = 2 5 ) neighboring pixel, the 64-th (n = 2 6 ) neighboring pixel 
up to the (N1/2)-th neighboring pixel. As a result, a sum of all the x direction moments, 1 • 1(1 ,y) + 2 • l(2,y) + ... + N1 

10 • |(N1 ,y), in each row is obtained in the processing element 400(1 ,y) at the leading edge of the corresponding row 
[0119] Next, j is initially set in S1 032 to zero (0). In S1 034 - S1040, transfer/adding process is performed so that an 
added result at each processing element Is added with another added result in another processing element 400(x, 
y+m) (in this case, 400(x, y+1)) that is located at a position (x, y+m) (where m = 2i, in this case, m = 2> = 1 ) adjacent 
to the subject pixel position in the y direction. 

is [0120] Next, while successively incrementing j by one (S1044), the processings of S1046 - S1040 are repeated. 
Thus, transfer/addition operation is repeated through the second (m = 2 1 ) neighboring pixel, the fourth (n = 2 s ) neigh- 
boring pixel, the eighth (n = 2 3 ) neighboring pixel, the sixteenth (n = 2*) neighboring pixel, the 32-th (n = 2 5 ) neighboring 
pixel, the 64-th (n = 2 6 ) neighboring pixel, ... up to the (N2/2)-th neighboring pixel. As a result, a sum Six* l(x,y) of 
the x direction moments of all the pixels is obtained in the processing element 400(1 ,1 ) at the leading end position of 

20 the processing element array 40. Then, the sum calculation process (S420) is ended. 

[0121] When the sum calculation process (S420) is completed, the program proceeds to S430 as shown in Fig. 11 . 
[0122] In S430, a pixel selection process is executed to select the leading end pixel position (1 , 1). 
[0123] During the pixel selection process, the process the same as that described with reference to Fig. 10(C) is 
performed. As a result, in S1 1 1 4 - S11 1 6, the multiplied result (1 ) is set only at the leading end processing element 400 

25 (1 f 1), while the multiplied result (0) is set at all the other remaining processing elements 400. 

[0124] When the pixel selection process (S430) is completed, the program proceeds to S432 as shown in Fig. 11 . 
In S432, the multiplied result obtained in the pixel selecting process (S430) is transferred from the register matrix 401 
to the A latch 402. Next, in S434, the calculated result finally obtained in the total sum calculation process (S420) is 
transferred from the register matrix 401 to the B latch 403. Next, the ALU 404 multiplies the value in A latch 402 and 

30 the value in B latch 403 in S436. As a result, at the processing element 400 of the leading end address (1 , 1), an x 
direction moment sum £ £ x • l(x,y) is obtained as a multiplied result of the calculated total sum £ £ x • l(x,y) and the 
value (1 ). On the other hand, at the other remaining processing elements 400, a zero value (0) is obtained as a multiplied 
result of the calculated total sum and the value (0). In S438, the multiplied result is stored in the register matrix 401 . 
In S440, the multiplied result is outputted from the output address "28" of the register matrix 401 to the output bus 1 55. 

35 As a result, the multiplied results, which are obtained at all the processing elements in S436, are outputted to the single 
output bus 155. In this case, only the processing element 400(1, 1) at the leading end position outputs the x direction 
moment sum IZx» l(x,y) as the multiplied result, while the other remaining processing elements output zero values 
(0) as the multiplied results. Accordingly, the output bus 155 transfers, to the control circuit 1 5, the x direction moment 
sum IZx« l(x,y), which is the logical OR value of the results calculated at all the processing elements 400. 

40 [0125] Next, in S450, the CPU 150 divides the received x direction moment sum £ £x • l(x,y) by the image intensity 
sum £ £ l(x,y), thereby obtaining an x coordinate of the center of gravity. 

[0126] Next, the program proceeds to S500. In S500, the y coordinate of the center of gravity is calculated in the 
same manner as described above for obtaining the x coordinate of the center of gravity. 

[0127] More specifically, as shown in Fig. 12, first, in S502, the CPU 150 reads, from the memory 151 , y-direction 
45 positional information (y address data "y") of each pixel. The CPU 150 transfers the y-direction positional information 
to the processing elements 400 so as to supply each processing element 400 with its y-directional positional information . 
This transfer operation is performed by temporarily storing each y address data "y" in the y direction data buffer 20, 
and then transferring the y address data via a corresponding data line 1 80y in the y direction data bus 1 8. The y address 
data is transferred successively from its least significant bit one bit at a time in a bit serial manner. In this case, the 
so same y address data "y" is assigned for all the pixels that are connected to one y direction data line 1 80y. The y address 
data is data of a binary number that is indicative of either one of values of 1 to N2. Accordingly, each data line 180y 
transfers the log 2 (N2) number of bits that constitute the binary number data indicative of the corresponding y address 
"y". Each data line 1 80y transfers the log 2 (N2) number of bits one bit at a time from the least significant bit. Thus, each 
data line 1 80y can transfer the y address data to all the processing elements 400(x,y) that are located at the corre- 
55 sponding y address. 

[0128] In each processing element 400(x,y), the transferred address data "y" is stored in the register matrix 401 in 
S504. Thereafter, the address data "y" is read into the B latch 403 from its least significant bit in S506. Image data I 
(x.y) is again read from the register matrix 401 into the A latch 402 in S508. The ALU 404 multiplies data in A latch 
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402 and data in B latch 403 in S510. The calculated result, that is, the y direction moment value y - l(x,y), is stored in 
the register matrix in S51 2. 

[0129] Next, in S520, a total sum calculation process is executed to determine a total sum of the y direction moments 
y • l(x,y) of all the pixels. 

5 [0130] During the total sum calculation process (S520), the sum calculation process, which is already described with 
reference to Fig. 1 0(B), is executed by setting the y direction moment value y ♦ l(x,y) as objective data. 
[0131] More specifically, in S1 002, at each processing element, objective data (a y direction moment value y • l(x, 
y), for which a sum is desired to be determined) is transferred from the register matrix 401 to the A latch 402. Next, in 
S1 004, each processing element 400(x,y) receives objective data (in this example, y • I (x+1 , y)) from another process- 

10 jng element 400(x+1 , y) that is located at an adjacent pixel position (x+1 , y) in the x direction. Next, in S1 006, the y- 
direction moment value y • l(x+1 , y) is stored into the B latch. By adding data in A latch 402 and data in B latch 403, y 
direction moment values in the two adjacent pixels are added together, and the added result is stored in the A latch 
402 via the register matrix 401 in S1008 - S1012. In this way, each pixel 400(x,y) stores therein a sum (y • l(x,y) + y • 
I(x+1 , y)) of y direction moment data of two adjacent pixels. 

15 [0132J Next, i is initially set in S1014to one (1). In S1018 - S1026, transfer/adding process is performed so that the 
calculated result (y • l(x,y) + y • l(x+1 , y)) at each processing element is added with another calculated result (y • I 
(x+2,y) + y • l(x+3, y)) that is stored in another processing element 400(x+2, y) that is located at a position (x+n, y) 
(where n = 2», in this case, n = 2 1 = 2) second closest to the subject pixel position in the x direction. 
[0133] Next, while successively incrementing i by one (S1028), the processings of S1018 - S1030 are repeated. 

20 Thus, transfer/addition operation is repeated through the fourth (n = 2 2 ) neighboring pixel, the eighth (n = 2 3 ) neighboring 
pixel, the sixteenth (n = 2 4 ) neighboring pixel, the 32-th (n = 2 5 ) neighboring pixel, the 64-th (n = 2 6 ) neighboring pixel, ... 
up to the (N1/2)-th neighboring pixel. As a result, a sum of all the y direction moments, y • 1(1 ,y) + y ♦ l(2,y) + ... + y I 
(N1 ,y), in each row is obtained in the processing element 400(1, y) at the leading edge of the corresponding row. 
[0134] Next, j is initially set in S1032 to zero (0). In S1 034 - S1040, transfer/adding process is performed so that an 

25 added result obtained at each processing element is added with another added result at another processing element 
400(x, y+m) (in this case, 400(x : y+1)) that is located at a position (x, y+m) (where m = 2', in this case, m = 2> = 1) 
adjacent to the subject pixel position in the y direction. As a result, the leading edge processing element 400(1 ,y), 
which is located at the leading edge position on each row, obtains a sum { [y • I (1 ,y) + y • I (2,y) + ... + y • I (N1 ,y)] + 
[ (v+1) • 1(1 »y +1 ) + (y +1 ) • 1 (2,y+1) + ... + (y+1) • I(N1 ,y+1) ] } of the added values obtained in two adjacent rows. 

30 [0135] Next, while successively incrementing j by one (S1044), the processings of S1046 - S1040 are repeated. 
Thus, transfer/addition operation is repeated through the second (m = 2 1 ) neighboring pixel, the fourth (n = 2^ neigh- 
boring pixel, the eighth (n = 2 3 ) neighboring pixel, the sixteenth (n = 2*) neighboring pixel, the 32-th (n = 2 5 ) neighboring 
pixel, the 64-th (n = 2 6 ) neighboring pixel, ... up to the (N2/2)-th neighboring pixel. As a result, a sum 2 X y • l(x,y) of 
the y direction moments of all the pixels is obtained in the processing element 400(1 ,1 ) at the leading end position of 

35 the processing element array 40. Then, the sum calculation process (S520) is ended. 

[0136] When the sum calculation process (S520) is completed, the program proceeds to S530 as shown in Fig. 12. 
[0137] In S530, a pixel selection process is executed to select the leading end pixel position (1,1). 
[0138] During the pixel selection process, the process the same as that described with reference to Fig. 10(C) is 
performed. As a result, inS1114-S1116, the multiplied result (1 ) is set only at the leading end processing element 400 

^o (1 ( 1), while the multiplied result (0) is set at all the other remaining processing elements 400. 

[0139] When the pixel selection process (S530) is completed, the program proceeds to S532 as shown in Fig. 12. 
In S532, the multiplied result obtained during the pixel selecting process (S530) is transferred from the register matrix 
401 to the A latch 402. Next, in S534, the calculated result finally obtained in the total sum calculation process (S520) 
is transferred from the register matrix 401 to the B latch 403. Next, the ALU 404 multiplies the value in A latch 402 and 

45 the value in B latch 403 in S536. As a result, at the processing element 400 of the leading end address (1 , 1), an y 
direction moment sum Ily • l(x,y) is obtained as a multiplied result of the calculated total sum Z X y • l(x,y) and the 
value (1). On the other hand, at the other remaining processing elements, a zero value (0) is obtained as a multiplied 
result of the calculated total sum and the value (0). In S538, the multiplied result is stored in the register matrix 401 . 
In S540, the multiplied result is outputted from the output address "28" of the register matrix 401 to the output bus 1 55. 

50 As a result : the multiplied results obtained at all the processing elements in S536 are outputted to the single output 
bus 155. In this case, only the processing element 400(1 , 1 ) at the leading end position outputs the y direction moment 
sum £ 2 y • l(x,y) as the multiplied result, while the other remaining processing elements output zero values (0) as the 
multiplied results. Accordingly, the output bus 155 transfers, to the control circuit 15, the y direction moment sum Z Zy 
• K x >y)> which is the logical OR value of the results calculated at alt the processing elements 400. 

55 [0140] Next, in S550, the CPU 1 50 divides the received y direction moment sum Z Z y ♦ l(x,y) by the image intensity 
sum E Z l(x,y), thereby obtaining a x coordinate of the center of gravity. 
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(Example 2) 

[0141] The parallel processing method employed by the present system is called SIMD, and is for conducting equiv- 
alent processings at all the processing elements. It is possible to attain a more flexible processing by allowing respective 
5 pixels to perform different processings or by allowing only a certain pixel to perform a special processing. According 
to the present embodiment, by using the data buses 1 7 and 1 8, it is possible to allow the respective processing elements 
400 to perform different processings. This is because it is possible to transfer calculation-control data to the respective 
processing elements 400 via the x direction data bus 1 7 and the y direction data bus 1 8. 

[0142] For example, when it is desired to allow only a certain pixel to perform a special processing, the objective 
10 pixel ((x1 , y1 ), for example) is designated by the x direction data bus 1 7 and the y direction data bus 1 8. That is, data 
(1) is transferred through the lines of (x1 . y1), while data (0) is transferred through the other remaining lines. Each 
processing element 400 multiplies the received data to data already retained therein, before performing image process- 
ing operation. As a result, only the processing element at position (x1, y1) can perform image processing operation. 
[0143] More specifically, a desired position processing (S600) is performed as shown in Fig. 13 during the image 
15 processing step S1 10 of Fig. 8. 

[0144] It is noted that when the desired position processing (S600) is started, image data l(x, y) at each pixel of the 
input image D has already been transferred in S104 (Fig. 8) from the corresponding transfer shift register 41 0 to the 
register matrix 401 of the corresponding processing element 400(x,y), and is being stored in some region in the register 
matrix 401 . 

20 [0145] During the desired position processing (S600), first, in S610, a pixel selection process is performed to select 
a desired pixel position (x1 , y1 ). During the pixel selection process (S61 0), the pixel selection process described already 
with reference to Fig. 10(C) is executed to select the desired pixel position (x1 , y1). 

[0146] More specifically, in S1102 - S1106, the CPU 150 transfers data (1) to an x-direction data line 170x (x = x1) 
with its x address being the same as the x address x1 of the objective pixel position (x1 , y1). The CPU 150 transfers 

25 data (0) to all the other remaining x-direction data lines 1 70x (x * x1 ). In this way, among all the processing elements 
400, only the processing elements 400(x1 , y) (1 ^ y ^ N2), whose x addresses are x1 , are set with data (1 ). The other 
remaining processing elements 400 are set with data (0). Next, in S1108 - S1112, the CPU 150 transfers data (1) to a 
y-direction data line 1 80y (y = y1 ) with its y address being the same as the y address y1 of the objective pixel position 
(x1, y1). The CPU 150 transfers data (0) to all the other remaining y-direction data lines 180y (y * y1). in this way, 

30 among all the processing elements 400, only the processing elements 400(x, y1 ) (1 ^ x ^N1 ), whose y addresses are 
y1, are set with data (1). The other remaining processing elements 400 are set with data (0). 

[01 47] Next, in S1 1 1 4, each processing element 400(x,y) is controlled to multiply data transferred from the x direction 
data line 170x and data transferred from the y direction data line 180y. The multiplied result is stored into the register 
matrix 401 in S1 1 1 6. Thus, the pixel selection process (S61 0) is completed. As a result, a multiplied result (1 ) is obtained 
35 only in the processing element 400(x1 , y1) at the desired address (x1 , y1). A multiplied result (0) is obtained in all the 
other processing elements 400. 

[0148] When the pixel selecting process (S610) is completed, the program proceeds to S612 (Fig. 13). In S612, the 
multiplied result obtained during the pixel selecting process (S61 0) is transferred from the register matrix 401 to the A 
latch 402. Next, in S61 4, objective data (image data l(x,y), for example) desired to be subjected to a calculation process- 

40 ing is transferred from the register matrix 401 to the B latch 403. Next, the value in A latch 402 and the value in B latch 
403 are multiplied in S616. As a result, at the processing element 400 of the objective pixel position (x1, y1), the 
objective data is obtained as it is. On the other hand, at the other remaining processing elements 400, zero (0) data 
is obtained. In S61 8, the multiplied result is stored in the register matrix 401 . Thereafter, in S620, a desired processing 
is performed onto the multiplied result. More specifically, the multiplied result obtained in S61 6 is transferred from the 

45 register matrix 401 to the A latch 402 or the B latch 403. Then, the ALU 404 performs a predetermined processing. As 
a result, only at the objective pixel position (x1, y1), processing is performed onto the objective data. At the other 
remaining pixel positions, processing is performed onto the zero (0) data. Results obtained by the processings are 
outputted to the control circuit 15 via the output bus 155. 

[0149] It is noted that in S620, calculation may not be performed. However, the multiplied result obtained in S616 
50 may be directly outputted as it is from the output address "28" to the control circuit 15 via the output bus 155. In this 
case, the parallel processing system 14 can output only desired data (image data l(x,y), for example), which is obtained 
at the desired address (x1 , y1), to the control circuit 15. 

[01 50] According to the desired position processing (S600), it is possible to allow the respective processing elements 
to perform different processings, by successively selecting the respective processing elements (pixels) while changing 
55 the contents of the processing operations to be executed in S620. 

[0151] Additionally, it is possible to select a plurality of pixels, rather than selecting only a single pixel, by transferring 
data (1) to x direction data lines 170x and y direction data lines 180y that correspond to the x addresses and the y 
addresses of the plural pixels and by transferring data (0) to other remaining data lines. Only at the selected plural 
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pixels, the multiplied results obtained in S1114 (Fig. 10(C)) become 1, and as a result, the objective data is obtained 
in S61 6 (Fig. 1 3) as the multiplied data. Accordingly, in S620, the parallel processing system 1 4 can perform a desired 
processing onto the objective data at the selected plural pixels. 

[0152] As the calculation processing performed in S620, various processing operations, such as an edge-enhance- 
5 ment operation to be described later, can be performed. 

(Example 3) 

[0153] By using the data buses 17 and 18, it is possible to retrieve a content (calculated result) from the register 
10 matrix 401 of a certain processing element 400 after all the processing elements perform predetermined calculations. 
[0154] In more concrete terms, a calculating-outputting process (S650)can be performed as shown in Fig. 14 during 
the image processing procedure S110 of Fig. 8. 

[0155] It is noted that when the calculating-outputting process (S600) is started, image data l(x, y) at each pixel of 
the input image D has already been transferred in S104 (Fig. 8) from the corresponding transfer shift register 410 to 
is the register matrix 401 of the corresponding processing element 400(x,y), and is being stored in some region in the 
register matrix 401 . 

[0156] During the calculating-outputting process (S650), first, in S660, each processing element 400 is controlled to 
perform a desired calculation onto image data l(x,y). As the desired calculation, various processings, such as the edge- 
enhancement process, can be employed. 
20 [0157] Next, a pixel selecting process (S680) is conducted to select an objective pixel position (x1 , y1). During the 
pixel selecting process (S680), the pixel selecting process that is described already with reference to Fig. 10(C) is 
executed onto the objective pixel (x1 , y1 ). As a result, data (1 ) is set as a multiplied result of S1 1 1 4 only at the objective 
processing element 400(x1 , y1 ), and data (0) is set as a multiplied result of S1114 at all the other remaining processing 
elements 400. 

25 [0158] When the selecting process (S680) is completed, the program proceeds to S682 (Fig. 14). In S682, the mul- 
tiplied result obtained during the pixel selecting process (S680) is transferred from the register matrix 401 to the A latch 
402. Next, the objective data (calculated result in S660), which is desired to be outputted, is transferred from the register 
matrix 401 to the B latch 403 in S684. Then, the value in A latch 402 and the value in B latch are multiplied in S686. 
As a result, the calculated result is obtained as it is in the processing element 400 only at the objective pixel position 

30 (x1 , y1 ). Data of (0) is obtained at the other remaining processing elements 400. After the multiplied result is stored in 
the register matrix 401 in S688, the multiplied result is outputted in S690 from the output address "28" to the output 
bus 155. The multiplied results of S686 are outputted to the output bus 155 from all the processing elements 400. 
Accordingly, the calculated result at the objective pixel 400(x1 , y1) is outputted to the control circuit 15 as a logical OR 
value of the multiplied results obtained at all the processing elements. 

35 

(Example 4) 

[01 59] It is possible to use a bi-directional data bus, rather than a one-directional data bus, as each of the x direction 
data bus 1 7 and the y direction data bus 1 8. The one-directional data bus has only a transfer/writing function to transfer 

40 data from the control circuit 15 to the processing elements 400 and to write the data to the processing elements 400. 
The bi-directiona! data bus has both of: the transfer/writing function to transfer data from the control circuit 15 to the 
processing elements 400 and write data into the processing elements 400, and an output/transferring function for 
outputting data from the processing elements 400 and transferring the outputted data from the processing elements 
400 to the control circuit 15. By using the addresses 30 and 31 of the register matrix also as outputs for they direction 

45 data line 180y and the x direction data line 170x, results calculated by the processing element 400 can be outputted 
via the y direction data line 180y or the x direction data line 170x to the control circuit 15. 

[01 60] By using the bi-directional data buses as the data buses 1 7 and 1 8 and using them bi-directionally, the control 
circuit 15 can receive a signal from some processing element 400 that is located at a certain position in the x and y 
directions. By using this function, the control circuit 15 can search some image (ml x m2 pixels) under investigation 
so through a parallel processing procedure. That is, the control circuit 15 can confirm within a short period of time the 
position where a matching signal is obtained. 

[0161] One example of the image searching operation will be described below. 

[0162] This searching operation is for determining whether a search pattern P is located within an input image D (= 
l(x>y)> where 1 ^ x ^ N1, 1 ^ y ^ N2). The searching operation is further for, when it is determined that the search 
55 pattern P is located within the input image D, detecting a position where the search pattern P is located. Many algorithms 
have been reported as examples of a matching algorithm. In this example, one of the most simple algorithms is em- 
ployed. This algorithm determines that two images are identical when a distance "ERROR(p,q)", represented by the 
following formula, between the two images is smaller than a threshold value. Here, (p,q) is a position of a reference 
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pixel (x,y) within the image D. 



5 



ERROH(p, q) 




P + ir q + j) ~ jj 



(2) 



[0163] Fig. 15 shows a flowchart of a searching process (S700), which is established based on this algorithm. The 

10 flow of data processes will be described below with reference to this drawing. 

[0164] It is noted that the searching process (S700) is executed during the image processing step S110 of Fig. 8. 
Each pixel (x,y) is defined as a reference position (p,q) for the image searching operation. The image data P(i,j) (1 ^ 
i§m1,1 ^ j ^ m2) of the search image P is stored in the memory 151 of the control circuit 15. 
[0165] It is also noted that when the searching process (S700) is started, image data l(x, y) (which will be referred 

15 to as "l(p, q)" hereinafter) at each pixel in the input image D has already been transferred in S104 (Fig. 8) from the 
corresponding transfer shift register 410 to the register matrix 401 of the corresponding processing element 400 (x,y) 
(which will be referred to as "400(p, q)" hereinafter), and is being stored in some region in the register matrix 401 . 
[0166] During the search process (S700), the CPU 150 first performs an initial -setting operation in S702. More spe- 
cifically, a leading end position (1 , 1 ) is set to a matching detection position (i, j) within the search pattern P. A calculated 

20 result Er(p, q), to be stored in the register matrix 401 of each processing element 400(p, q), is reset. 

[0167] In S703, image data P(i,j), at the present detecting position (ij) (which is now a leading end position (1 , 1)) 
within the search pattern P, is read out from the memory 151 . The image data is transmitted via the* data bus 17, 18 
to each processing element 400. 

[0168] In S704, each processing element 400 calculates an absolute value of a difference between image data l(p, 
25 q), at the subject pixel, which is stored in some region of the register matrix, and image data P(i, j) at the matching 
detection position of the search pattern. The processing element 400 adds the calculated absolute value to the value 
Er(p,q). More specifically, the absolute value of the difference between the image data l(p,q) and the image data P 
(1 ,1 ), at the leading end position of the search image, is first calculated. Then, another region is assigned in the register 
matrix 401 , and the calculated absolute value is stored as a value Er(p,q) in that region. 
30 [0169] In S705, the CPU 150 judges whether the detection position (i, j) reaches the final position (ml , m2). In this 
case, the detection position (i s j) is still at the leading end position (1 , 1), but does not reach the final position (no in 
S705). Accordingly, the program proceeds to S706. In S706, when i * ml , only the value i is incremented by one (1). 
When i = ml , it is determined that the detection position reaches the trailing end in the x direction. Thus, i is reset to 
one (1), and j is incremented by one (1). In this case, i = 1 andj = 1 , and therefore i and j are renewed into i = 2 andj 
35 = 1 . in S707, image data at position (p+i-1 , q+j-1) (in this case, image data l(p+1 ,q) at position (p+1 ,q)), is transferred 
to the register matrix 401 of the processing element 400(p,q). More specifically, by using the transferring function of 
the register matrix 401 to its neighboring register matrix 401 , image data value l(p+1 , q), which is stored in the neigh- 
boring pixel 400(p+1 ,q), is transferred to a vacant region in the subject pixel 400(p,q). 

[0170] When the transfer operation is completed, the program returns to S703, in which the CPU 1 50 retrieves, from 
40 the memory 151 , image data P(i, j) at the present location (i, j) (in this case, (2,1)) within the search pattern P. The 
CPU 150 transfers the retrieved image data to each processing element 400(p,q) via the data bus 17, 18. In S704, 
each processing element 400(p, q) calculates the absolute value of a difference between the image data P(i, j), at the 
present matching detection position of the search pattern, and the image data l(p+1, q) , at a pixel adjacent to the 
subject processing element 400(p,q). It is noted that the image data l(p+1 , q) is presently being stored in the register 
45 matrix 401 of the processing element 400(p,q). The processing element 400(p, q) adds the calculated absolute value 
to the value Er(p,q). Because the detection position (i j) does not yet reach the final position (ml ,m2) (no in S705), the 
program proceeds to S706, and the detection position (i,j) is renewed. 

[0171] The above-described processes (S703 - S707) are repeated until the detection position (i,j) reaches the final 
position (ml ,m2). 

so [0172] By repeatedly performing the adding operation in S704 in this way, each processing element 400(p, q) can 
obtain a value ERROR(p,q). 

[0173] It is noted that the transfer operation in S707 may be performed by successively transferring data to neigh- 
boring pixels in x and y directions. 

[0174] When the detection position reaches the final position (ml ,m2) and when the value ERROR(p,q) is obtained, 
55 the program proceeds from S705 (yes in S705) to S708. In S708, the CPU 150 sets a threshold value E^. In S709, 
the CPU 150 transfers the set threshold value E^ to each processing element 400 via the data bus 17 or 18. 
[0175] In S710, each processing element 400(p,q) compares the calculated value ERROR(p,q), that is, Er(p,q) with 
the threshold E th . More specifically, each processing element 400(p,q) calculates a value of "ERROR(p,q) - threshold 
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value E th ", and outputs, as a compared result, a bit of a sign bit (positive or negative sign) of the calculated result. That 
is, when Er(p,q) is smaller than or equal to threshold value E th! the program proceeds to S711 , and outputs one (1). 
When Er(p,q) is greater than threshold value E th , the program proceeds to S712, and outputs zero (0). Each processing 
element 400 outputs the output data to both of the data buses 1 7 and 1 8. The CPU 1 50 calculates a total sum of output 
5 signals transferred from the data buses 17 and 18, thereby counting the total number of processing elements that 
output one (1). 

[0176] In S713, the CPU 150 judges the counted number. When the counted number is zero (0), the program pro- 
ceeds to S714, and it is confirmed that there is no image matching. 

[0177] When the counted number is one (1), only a single data line 170x and only a single data line 180y output 
10 signals (1) among the data buses 17 and 18. Accordingly, in S715, the CPU 150 determines positional data (p,q) of 
the processing element 400 that outputs one (1), based on the positions x and y of the output signal lines 170x and 
1 80y that output signals (1 ). 

[0178] It is noted that the output bus 155 may be used instead of the data buses 1 7 and 18. That is, each processing 
element 400 can be controlled to multiply an output (0 or 1 : a judged result in S71 0) by its positional data (p,q) and to 

is output the multiplied result to the output bus 155. In this case, only a pixel (matching position pixel), that outputs an 
output signal of one (1) as a judging result of S710, outputs the positional data (p,q). All the other remaining pixels 
output values (0) that are the products of their output- signals (0) and the positional data (p,q). Accordingly, only the 
positional data (p,q) of the matching position will be transferred to the control circuit 1 5 via the output bus 1 55. According 
to this method, it is possible to transfer, to the control circuit 15, not only the matching position data (p,q) but also other 

20 matching calculated results, such as the absolute value of the difference between the input image D and the search 
pattern P. 

[0179] On the other hand, when the counted number is two or more, the threshold E^ is decreased in S716, and 
then the program returns to S709. Thus, the searching range is successively decreased until the counted number 
becomes one (1). 

25 [01 80] As described above, the high-speed vision sensor 1 0 of the present embodiment includes the analog-to-digital 
converter array 13 and the parallel processing system 1 4. The analog-to-digital converter array 1 3 has one analog-to- 
digital converter 210 in correspondence with all the photodetectors 120 that belong to each row of the photodetector 
array 1 1 In the parallel processing system 14, the processing elements 400 are provided in one to one correspondence 
with the photodetectors 120, and the shift registers 410 are provided also in one to one correspondence with the 

30 photodetectors 120. The high-speed vision sensor 10 further includes data buses 17 and 18 and data buffers 19 and 
20 for performing data transferring operation to the processing elements 400. The processing elements 400 can perform 
high-speed image processings between their neighboring pixels by parallel processing. Further, by using the data 
buses 17 and 18, it is possible to perform, at a high speed, even such a calculation processing that requires data to 
be transmitted from outside. 

35 [0181] According to the high speed vision sensor 1 0 of the present embodiment, during the image processing step 
S110, before achieving the above-described image processes (Figs. 9 - 15), a preprocessing can be performed to 
attain various image processings onto the input image l(x,y). 

[0182] For example, an "edge-enhancement" can be performed. It is noted that ,, edge-enhancement ,, is one of the 
processes that are used most frequently in image processing. In order to calculate the edge-enhancement in one of 
*o the simplest manners, a two-neighboring edge-enhancement is employed to calculate a difference between image 
intensity at each pixel and image intensity at a neighboring pixel on the left side thereof. 

[0183] More specifically, if the input image intensity at a position (x, y) is l(x, y), then image intensity data l'(x, y) of 
a two-neighboring edge-enhancement image can be expressed by the following equation: 

l'(x,y) = ll(x,y)-l(x-1,y)l. 

[0184] In order to perform a two-neighboring edge-enhancement operation as a preprocess during the image 
processing step of S110, a two-neighboring edge-enhancement step (S1200) is executed as shown in Fig. 16 before 

so the processes of S300, S600, S650, or S700 in the (Example 1) - (Example 4). 

[0185] More specifically, first, in S121 0, each processing element 400(x,y) receives data l(x-1 , y) from its left neigh- 
boring pixel. That is, the processing element 400(x,y) receives the data l(x-1 , y) at its input terminal for the four neigh- 
boring pixels. The data l(x-1 , y) is stored in the register matrix 401 of the processing element 400(x, y). Next, in S1211 , 
data l(x, y) and data l(x-1 , y) are transferred from the register matrix 401 into the A latch 402 and B latch 403, respec- 

55 tively. In S1212, the ALU 404 calculates the difference between these two values l(x, y) and l(x-1 , y). In S1213, the 
result of the calculation is stored temporarily in the register matrix 401 . After this calculation is completed, the result 
of the calculation is read again into the A latch 402 in S1214. In S1215, the ALU 404 calculates the absolute value of 
the difference. In S121 6, the result of this calculation is stored in the register matrix 401 . 
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[0186] The thus obtained two-neighboring edge-enhanced image l'(x,y) is subjected to the image processing step 
(S300, S600, S650, or S700) in (Example 1) - (Example 4). 

[0187] It is noted that when performing a four-neighboring edge-enhancement, if the intensity of an input image at 
point (x, y) is represented by l(x, y), the image intensity l'(x, y) at point (x, y) of a four-neighboring edge-enhancement 
5 image can be represented by the following equation: 

!'(*. y) = Kx, y-1 ) + l(x, y+1 ) + l(x-1 , y) + l(x+1 , y) - 4l(x, y) . 

10 [0188] When performing a four-neighboring smoothing, the image intensity value {'(x, y) of a four-neighboring 
smoothed image is represented by the following equation: 

l'(x, y) = (4l(x, y) + l(x-1 , y) + l(x+1 , y) + l(x, y-1 ) + l(x, y+1 )) / 8. 

15 

[0189] Table 1 below shows a list showing the number of steps required for performing each of several algorithms, 
which include the above-described algorithms and other algorithms and which are frequently used in image processing 
operations. The Table 1 also shows the length of a processing time required for completing each algorithm when the 
subject algorithm is executed according to the present embodiment. 

20 

Table 1 



Name of Process 


Number of steps 


Time required (\is) 


Two- neigh bo ring edge detection 


1 -bit input/ output 


5 


0.40 


Four-neighboring edge detection 


1 -bit input/ output 


11 


0.72 


Four-neighboring smoothing 


1 -bit input/ output 


14 


1.0 


Four-neighboring edge detection 


8-bit input/ output 


70 


5.6 


Four-neighboring edge detection 


8-bit input/ output: 


96 


7.7 


Four-neighboring thinning 


1 -bit input/ output 


23 


1.9 


Eight-neighboring thinning 


1 -bit input/ output 


53 


4.2 


Convolution 


1 -bit input, 4-bit output 


40 


3.2 


Convolution 


4-bit input, 11 -bit output 


372 


30 


Poisson's equation 


1-bit input, 8-bit output 


63 


5.0 



[0190] As can be apparent from Table 1 , according to the present embodiment, genera! image processings, such as 
40 smoothing, thinning, convolution, correlation, and masking, can be attained extremely rapidly by executing a complete 
parallel processing. Accordingly, the present embodiment can be applied to such fields, as factory automation (FA) 
robot control, contrary to conventional vision sensor devices which have slow image processing speeds and slow 
transfer rates. 

[0191] It is noted that the length of each calculation time listed in Table 1 does not include the time required by the 
45 shift registers 410 to transfer image data. It is also noted that the data transfer rate is limited by the conversion speed 
of the analog-to-digital converter 210. If the length of time required for analog-to-digital conversion is 1 fi s/bit, for 
example, then the time, required for transferring 128 x 128 pixels x 8 bits' worth of image data in parallel using the 128 
rows, would be 128 pixels x 8 bits x 1 |x s/bit = 1 ms. According to the present embodiment, this transfer process is 
executed in parallel with the calculation processes. 
so [0192] The object of the present embodiment is to provide an image processing system of a practically high speed 
and sufficiently high resolution. It is noted that the robot control in FA systems requires such a resolution that 128 x 
128 photodetectors 120 or more are arranged. According to the present embodiment, the photodetector array 1 1 and 
the parallel processing system 14 can be separated from each other. Accordingly, each device can be constructed with 
a high level of integration. It is therefore possible to attain the required high resolution. Additionally, the processing 
55 speed has to match the actuator speed of a robot (1 - 10 ms). According to the present embodiment, the processing 
speed is determined by the speed of the analog-to-digital conversion performed in the analog-to-digital converters 210. 
This conversion rate can be made sufficiently fast. 

[0193] For example, according to the present embodiment, the analog-to-digital conversion rate per pixel is 1 p. s/ 
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bit. It takes 0.768 ms (= 6 u. s x 1 28) to convert analog signals from all the 1 28 photodetectors 1 20 in one row into 6-bit 
digital signals (64 gradation), for example. It is noted that the processing elements are provided in one-to-one corre- 
spondence with the photodetector elements. All the processing elements are operated in parallel. Accordingly, most 
calculation processings can be completed within 0.4 ms as apparent from Table 1. Additionally, by performing the 
5 calculating processes and the transfer processes in parallel, it is possible to reduce the amount of dead time between 
the respective processes and to shorten the overall processing time. 

[0194] As described already, each analog-to-digital converter 210 performs analog-to-digital conversion beginning 
from the most significant bit. Accordingly, it is possible to change the gradation of the analog-to-digital conversion, by 
outputting a reset signal R after a desired number of bits have been converted so that the process will shift to perform 

1 o an analog-to-digital conversion of the next optical signal. It is therefore possible to perform complicated processes at 
a higher rate of speed. For example, in order to track an object which is moving at a high rate of speed, it is preferable 
to process images in two levels, that is, one bit. In this case, the transfer time can be shortened to 0.128 ms, which is 
about 1/6 the time required for transferring six bits. This method can therefore be applied to the control of high-speed 
feedback operation. Conversely, when the object is moving slowly, the gradation can be raised to be able to track the 

*s object with a higher precision. 

[0195] In order to allow the output from the analog-to-digital converter to have a variable bit length, it is necessary 
to adjust, before inputting the output to the shift registers, the varied bit length of the input to a fixed length. It is noted 
that a shift register line, for transferring data at a normal length of 8 bits, for example, is constructed from one set of 
shift registers that has the total fixed length equal to the product of eight bits and the total number (N1 ) of pixels in one 

20 row. In the one set of shift registers, a plurality of eight bit-shift registers, which are spaced from one another and each 
of which has eight bits, function as individual shift registers to transfer data for their corresponding pixels. Therefore, 
if the bit length of the image data for each pixel is not fixed to eight bits, then it is impossible to transfer each pixel's 
worth of image data correctly to the shift register at the corresponding position. For this reason, each pixel's worth of 
image data is added with a dummy signal to have eight bits in total before the image data is transferred to the shift 

25 registers. It can be ensured that image data will be transferred correctly. 

[01 96] As described above, the high-speed vision sensor 1 0 of the present embodiment includes the analog-to-digital 
converter array 13 and the parallel processing system 14. The analog-to-digital converter array 13 has one analog-to- 
digital converter 210 in correspondence with all the photodetectors 120 that belong to each row of the photodetector 
array 11 . In the parallel processing system 14, the processing elements 400 are provided in one to one correspondence 

30 with the photodetectors 120, and the shift registers 410 are provided also in one to one correspondence with the 
photodetectors 120. By providing a processing element 400 in correspondence with each photodetector 120, it is 
possible to perform, at a high speed, image processing calculations between neighboring pixels by parallel processing. 
[01 97] Additionally, by providing an analog-to-digital converter 21 0 in each row, it is possible to reduce the number 
of transfer lines, provided between the photodetectors 120 and the processing elements 400, in comparison to acon- 

35 figuration where an analog-to-digital converter 210 is provided in correspondence with each photodetector 120. The 
photodetectors 120 and the processing elements 400 can be produced separately, before being assembled together. 
Accordingly, both components can be manufactured with an optimal level of integration. Production of the high-speed 
vision sensor 10 having a large number of pixels can be simplified. It is noted that because one analog-to-digital 
converter 21 0 is provided for each row, the overall processing speed is limited by the A/D conversion process. However, 

40 most processes can be completed within one millisecond, even when 128 x 128 pixel images, which are sufficient for 
FA robot control, are processed at a 64 gradation. Such a high-speed processing can be attained. Accordingly, the 
high-speed vision sensor 1 0 of the present embodiment not only can be constructed with a simple circuit, but also can 
perform basic image calculations at a high speed. 

[01 98] Especially, in order to perform a processing that requires data supplied from outside, it is possible to efficiently 
45 perform data transfer operation (transmission/reception operation) by using the x direction data bus and the y direction 
data bus. It is therefore possible to perform high speed calculation. 

[0199] Further, by providing a shift register 41 0 in correspondence with each processing element 400, the transfer 
process can be performed independently from the calculation processing, thereby achieving calculation process and 
transfer process efficiently. Because the calculation process and the transfer process are executed in parallel, it is 
50 possible to reduce wait time between the respective processes and to perform image processing at a faster overall 
speed. That is : it is possible to attain a function to perform calculation processes and transfer processes independently 
from each other by using the transfer shift registers to transfer data from the analog-to-digital converters to the process- 
ing elements. A real-time operation can be attained. 

[0200] The high-speed vision sensor of the present invention can be modified in a variety of ways, and is not limited 
55 to the above-described embodiment. 

[0201] For example, in the above-described embodiment, the data buffers 19 and 20 are provided. However, if a 
sufficiently high transfer speed can be attained between the control circuit 15 and the data buses 17 and 18, it is 
unnecessary to provide the data buffers. 
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[0202] In the above-described embodiment, the data transfer from the analog-to-digital converters 21 0 to the process- 
ing elements 400 is performed by the transfer shift registers 41 0. However, the transfer shift registers 41 0 may not be 
provided. That is, as shown in Fig. 17, each analog-to-digital converter 21 0 may be connected to a register matrix 401 
in a processing element 400(1 , y) that is located at a leading edge of a corresponding row in the parallel processing 
5 system 14. In this case, pixel data l(x,y) outputted from the analog-to-digital converter 210 on each row is transmitted 
to a corresponding processing element 400(x,y) by being successively transmitted between the processing elements 
400(1 ,y) - 400(x,y) which are successively arranged in the x direction. 

[0203] If no shift registers 41 0 are provided, an extra amount of time would be required to transfer image data to the 
processing elements 400 in addition to the calculating time in Table 1 . It is noted that as described already, the analog- 

10 to-digital conversion rate per pixel is about 1 \i s/bit. Accordingly, it takes 0.768 ms = 6 u, s x 128 to convert analog 
input signals, outputted from all the 1 28 photo detectors 1 20 in one row, into 6-bit digital signals (64 gradation). On the 
other hand, most image processings can be completed within 0.4 ms as apparent from Table 1 . Accordingly, even 
when considering the transfer time, most image processes can be performed within one milliseconds. Thus, a suffi- 
ciently high speed can be attained. 

is [0204] In the above-described embodiment, the analog-to-digital converter 21 0 includes the charge amp 221 . How- 
ever, the analog-to-digital converter 210 and the charge amp 221 can be provided separately as shown in Fig. 18. As 
shown in this figure, an amp array 12 is connected to the photodetector array 11. The amp array 12 has N2 charge 
amps 221 . The analog-to-digital converter array 13 is provided between the amp array 12 and the parallel processing 
system 14. The analog-to-digital converter array 13 has N2 analog-to-digital converters 210, With this construction, 

20 each amp 220 in the amp array 12 successively converts charges, outputted from the N1 photodetectors 120 on the 
corresponding row 11 0 of the photodetector array 11 , into voltage signals. These analog voltage signals are outputted 
to the corresponding analog-to-digital converter 210 in the analog-to-digital converter array 13. The analog-to-digital 
converter 21 0 successively converts the analog voltage signals to digital signals, and supplies the digital signals to the 
parallel processing system 14. 

25 [0205] Additionally, in the above-described embodiment, in order to calculate: the total sum of image intensities, the 
total sum of x-direction moments, and the total sum of y-direction moments, the total sum calculation process of Fig. 
10(B) is executed. Instead, the output bus 155 can be added with a circuit for calculating a total sum of outputs from 
the processing elements 400. This circuit may be used to calculate the total sums. 

30 INDUSTRIAL APPLICABILITY 

[0206] The high-speed vision sensor of the present invention can be employed in a wide variety of vision recognition 
processes, including FA robot control. 

35 

Claims 

1. A high-speed vision sensor, comprising: 

40 a photodetector array having a plurality of photodetectors, which are arranged two-dimensionally in a plurality 

of rows and in a plurality of columns; 

an analog-to-digital converter array having a plurality of analog-to-digital converters which are arranged one- 
dimensionally such that each anaiog-to-digital converter corresponds to one row in the photodetector array, 
each analog-to-digital converter converting, into digital signals, analog signals which are successively output- 

45 ted from the photodetectors in the corresponding row; 

a parallel processing system including a parallel processing element array, the parallel processing element 
array having a plurality of processing elements which are arranged two-dimensionally in a plurality of rows 
and in a plurality of columns and in one-to-one correspondence with the plurality of photodetectors in the 
photodetector array, each processing element performing a predetermined calculation on digital signals which 

50 are transferred from the analog-to-digital converter array; 

a column-direction data-transfer bus including a plurality of column-direction data-transfer data lines which 
are arranged in one to one correspondence with the plurality of columns in the parallel processing system, 
each column-direction data-transfer data line being connected to the processing elements that are located in 
the corresponding column and performing data transfer operation with each processing element in the corre- 

55 sponding column; 

a row-direction data-transfer bus including a plurality of row-direction data-transfer data lines which are ar- 
ranged in one to one correspondence with the plurality of rows in the parallel processing system, each row- 
direction data-transfer data line being connected to the processing elements that are located in the corre- 
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spending row and performing data transfer operation with each processing element in the corresponding row; 
and 

a control circuit controlling the photodetector array, the analog-to-digital converter array, the parallel processing 
system, the column-direction data-transfer bus, and the row-direction data-transfer bus. 

A high-speed vision sensor as claimed in claim 1 , wherein the control circuit includes a center-of -gravity calculation 
control portion for controlling each column-direction data-transfer data line to perform data transfer operation to 
transfer positional information of the corresponding column to the processing elements on the corresponding col- 
umn, for controlling each row-direction data-transfer data line to perform data transfer operation to transfer posi- 
tional information of the corresponding row to the processing elements on the corresponding row, and for controlling 
each processing element to perform a predetermined calculation to calculate a center of gravity of the digital signals 
based on the received positional information of the corresponding row and of the corresponding column. 

A h igh-speed vision sensor as claimed in claim 1 , wherein the control circuit includes a control portion for controlling 
a predetermined processing element to perform a predetermined calculation onto the digital signals by controlling 
each column-direction data-transfer data line and each row-direction data-transfer data line to perform data transfer 
operation to transfer predetermined calculation-control data. 

A high-speed vision sensor as claimed in claim 1 , wherein the control circuit includes a data transfer control portion 
for controlling each column-direction data-transfer data line and each row-direction data-transfer data line to per- 
form data transfer operation to transfer predetermined calculation -control data, thereby allowing calculation result 
data, obtained at a processing element which is located at a corresponding row and a corresponding column, to 
be transferred to the control circuit. 

A high-speed vision sensor as claimed in claim 1 , further comprising data buffers which are provided in corre- 
spondence with the column-direction data-transfer bus and the row-direction data-transfer bus, respectively. 

A high-speed vision sensor as claimed in claim 1 , wherein the parallel processing system further includes a shift 
register array, the shift register array having a plurality of shift registers which are disposed in one-to-one corre- 
spondence with the plurality of analog-to-digital converters and in one-to-one correspondence with the plurality of 
rows of processing elements, each shift register successively transferring digital signals, which are outputted from 
the corresponding analog-to-digital converter and which are equivalent to signals outputted from the photodetec- 
tors in a corresponding photodetector row, to predetermined processing elements in the corresponding row. 
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FIG. 10 (A) 
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FIG. 10 (B) 
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FIG. 10 (C) 
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FIG. 11 
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FIG. 12 
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FIG. 13 
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FIG. 14 
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FIG. 16 
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FIG. 18 
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